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I. 


INTRODUCTION 


A.  BACKGROUND 

In  2002,  Section  801(d)  of  the  National  Defense  Authorization  Act  (NDAA) 
established  a  requirement  for  a  program  review  structure  for  services  procured  by  the 
Department  of  Defense  (DOD)  that  is  similar  to  the  procurement  of  weapon  systems  by 
the  DOD  (NDAA  of  FY  2002).  Since  then,  the  DOD  has  released  numerous  policies  to 
address  service  acquisitions  to  meet  the  intent  of  the  NDAA;  however,  oversight  at  the 
DOD  Component  level  was  not  being  conducted  as  it  was  expected  it  should.  As  a  result. 
Better  Buying  Power  (BBP)  1.0  included  a  requirement  to  create  a  Senior  (Services) 
Manager  (SSM)  for  each  DOD  Component  as  part  of  the  Improve  Tradecraft  in  Services 
Acquisition  tenant  (Department  of  Defense,  n.d.).  The  Army  Senior  Services  Manager 
(SSM)  is  required  to  ensure  proper  oversight  of  service  acquisitions,  which  is 
accomplished  by  establishing  an  Army  Services  Strategy  Panel  (ASSP)  review  and 
briefing  to  the  Deputy  Assistant  Secretary  of  the  Army  for  Procurement  (DASA(P))  SSM 
for  all  service  acquisitions  over  $250M  (Army  Federal  Acquisition  Regulation 
Supplement  [AFARS]  5137.590-5)  [previously  $500M];  Headquarters,  Department  of  the 
Army,  2010).  In  addition,  service  acquisitions  meeting  or  exceeding  $1B  must  participate 
in  Office  of  the  Secretary  of  Defense  (OSD)  Peer  Reviews  to  ensure  proper  oversight  is 
conducted  (USD[AT&L],  2008).  The  perceived  intent  of  this  was  to  treat  services 
acquisitions  like  the  Milestone  Decision  Authority  (MDA)  review  to  determine  true  need. 
However,  the  acquisition  of  services  is  not  comparable  to  the  development  of  products. 

B.  PURPOSE 

This  joint  applied  project  (JAP)  will  analyze  the  Army  Service  Acquisition 
Review  Requirements  and  the  perceived  effectiveness  of  intended  improvements. 

To  achieve  the  goal  of  this  JAP,  our  analysis  will  focus  on  two  main  areas:  policy 
and  the  process  of  creating  an  effective  survey  instrument.  The  study  examines  the  events 
leading  up  to  the  development  and  implementation  of  these  policies  through  documented 
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testimonies  and  reports.  The  end  result  of  this  study  is  to  provide  a  survey  that  can  be 
utilized  by  the  Army  to  assess  the  effectiveness  of  the  service  acquisition  review  process. 


C.  RESEARCH  QUESTIONS 

Since  this  requirement  has  been  established  for  Services  Acquisitions,  numerous 
policies  and  guidance  have  been  released  to  support  the  requirement.  Unfortunately,  to 
the  detriment  of  efficiency  and  reduced  bureaucracies,  when  compared  to  the  Better 
Buying  Power  tenants,  these  policies  and  guidance  are  perceived  to  be  contrary  to  the 
added  review  requirements  for  service  acquisition.  To  address  these  perceptions,  this 
research  seeks  to  provide  an  effective  survey  instrument  that  answers  the  following 
questions: 

1 .  Are  the  Army  Service  acquisition  requirements/reviews  leading  to 
desired  outcomes? 

2.  Does  the  value  added  through  completion  of  the  requirements/reviews 
exceed  the  resource  outputs  required  to  complete  the  process? 

3.  Is  proper  training/mentorship  available  prior  to  the  Army  service 
acquisition  requirements/reviews  to  maximize  the  value  of  the  ASSP? 

4.  Are  commands  capturing  metrics  to  document  the  requirements/ 
reviews  impacts? 

5.  Are  commands  taking  proper  measures  to  capture  lessons  learned? 

D.  SCOPE  AND  LIMITATIONS 

This  scope  of  this  project  focuses  on  policies  and  guidance  from  the  OSD,  the 
United  States  Army  (Army),  the  Government  Accountability  Office  (GAO),  and  BBP 
initiatives  specific  to  the  Army  as  it  pertains  to  Service  Acquisition  Requirements.  Time, 
resources,  and  willing  participants  precluded  this  research  from  assessing  a  larger  sample. 

E.  METHODOLOGY 

To  determine  the  applicability  of  reviewed  Army  specific  policies,  an  analysis 
was  conducted  to  identify  the  appropriate  DOD  and  Army  policies  and  guidance  needed 
to  create  an  effective  survey.  Upon  review  of  the  data,  an  analysis  was  performed  on  how 
to  prepare  effective  survey  questions  as  well  as  determine  what  information  would  be 
most  beneficial  to  collect.  No  official  survey  will  be  conducted  for  this  JAP,  instead  a 
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survey  field  test  will  be  distributed  to  individuals  that  have  submitted  requirements 
packages  through  the  ASSP  process.  The  survey  field  test  was  designed  to  determine  the 
appropriateness  of  the  survey  questions  and  answer  the  following: 

1 .  Are  the  survey  questions  clear  and  easy  to  understand? 

2.  Are  the  questions  free  of  bias? 

3.  Is  the  individual  taking  the  survey  comfortable  responding  to  the 
questions? 

The  final  step  involves  interpreting  the  results  of  the  data  to  assist  in  identifying 
any  potential  recommended  changes  or  improvements  to  guidance  found  to  be 
burdensome  and  non-value  added. 

F.  PROJECT  ORGANIZATION 

Chapter  I  -  Introduction.  This  chapter  presents  the  purpose,  research  questions, 
scope  and  limitations,  and  methodology  for  our  analysis. 

Chapter  II  -  Policy  Background.  This  chapter  provides  a  review  of  policies  and 
guidance  specific  to  the  United  States  Army  as  it  pertains  to  service  acquisition 
requirements  and  briefly  examines  the  events  leading  up  to  the  development  and 
implementation  of  these  policies  through  documented  testimonies  and  reports. 

Chapter  III  -  Policy  Impacts  Analysis.  This  chapter  analyzes  the  impacts  of  policy 
and  guidance  when  compared  to  the  BBP  tenants. 

Chapter  IV  -  Research  Methodology.  This  chapter  discusses  the  research 
methodology  used  to  gather  data. 

Chapter  V  -  Findings  and  Results.  This  chapter  provides  an  analysis  of  the 
information  gathered  from  the  study. 

Chapter  VI  -  Summary  and  Recommendations.  This  chapter  contains  an  overall 
summary  including  the  results  of  the  research  provided  by  the  survey  field  test  as  well  as 
recommended  changes  or  improvements. 
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II.  POLICY  BACKGROUND 


This  chapter  provides  a  review  of  the  policies  and  guidance  (OSD,  Army,  GAO 
and  BBP)  specific  to  the  Army  as  it  pertains  to  Service  Acquisition  Requirements.  The 
policies  addressed  only  reflect  general  services  acquisition  related  to  the  Military 
Components  and  do  not  provide  reference  to  DOD  Components  outside  the  Military 
Departments  (Defense  Agencies,  Field  Activities  and  Combatant  Commands), 
Information  Technology,  Research  and  Development  or  construction,  as  they  do  not 
apply  to  the  intent  of  this  discussion  on  Army  services  acquisition. 

In  his  2001  GAO  testimony,  David  E.  Cooper  testified  that  service  procurements 
were  not  being  done  efficiently.  Cooper  (2001)  stated,  “Agencies  are  not  clearly  defining 
their  requirements,  fully  considering  alternative  solutions,  performing  vigorous  price 
analyses,  or  adequately  overseeing  contractor  performance.  Further,  it  is  becoming 
increasingly  evident  that  agencies  are  at  risk  of  not  having  enough  of  the  right  people 
with  the  right  skills  to  manage  service  procurements”  (p.  1).  According  to  DOD’s  USD 
(AT&L),  although  there  was  an  improvement  in  the  quality  of  services  acquired,  DOD 
was  still  unable  to  meet  their  needs  for  higher  levels  of  excellence  (Brock,  2002).  In  2001 
Congress  required  that  DOD  improve  service  acquisition  planning,  tracking,  and 
oversight. 

Section  2330  of  title  10,  United  States  Code,  enacted  in  2001  and  amended 
in  2006,  requires  the  (USD(AT&L)  and  the  military  departments  to 
establish  a  management  structure  for  the  acquisition  of  services. 
USD(AT&L)  and  the  military  departments  established  an  initial 
management  structure  for  the  review  and  approval  of  service  acquisitions 
in  2002.  (DiNapoli,  2016,  p.  5). 
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A.  THE  OFFICE  OF  THE  SECRETARY  OF  DEFENSE  (OSD) 

Flow  of  Policy  is  as  follows: 


Figure  1 .  Flow  of  Policy 


The  first  OSD  service  acquisition  policy  was  released  in  FY  2002.  There  have 
been  numerous  policies,  memorandums,  and  instructions  released  from  the  OSD  since. 
The  latest  and  most  comprehensive  service  acquisition  instruction  was  the  release  of 
Department  of  Defense  Instruction  (DODI)  5000.74,  signed  January  2016.  This  policy 
has  been  in  the  making  for  years  and  flows  down  to  the  DOD  Components,  which  will 
necessitate  revisions  (or  creation)  of  their  policies  for  acquisition  of  services.  DODI 
5000.74  reflects  all  prior  requirements  that  have  been  pushed  down  from  the  OSD,  as  a 
result,  the  former  policies,  although  important  to  reflect  the  timeframe  of  requirements 
changes  for  service  acquisitions,  are  not  as  relevant.  Each  policy/memorandum/ 
instruction  is  summarized  in  Table  1. 
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Table  1.  Office  of  the  Secretary  of  Defense  Service  Acquisition  Policies 

Office  of  the  Secretary  of  Defense  Service  Acquisition  Policies 


Year 

DOD  Policy/Memorandum/ 
Instruction 

Details 

2002 

Section  801(d)  of  NDAA  FY  2002 

Public  Law  107-107  (author:  Congress) 

Requires  the  OSD  to  issue  and  implement  a 
policy  that  apphes  to  the  procurement  of 
services  by  the  DOD 

2002 

Acquisition  of  Services  policy 
(author:  USD[AT&L}) 

OSD  required  the  Military  Components  to 
submit  a  process  and  procedures  for 
management  and  oversight  of  all  service 
acquisitions  for  review  by  an  OSD  team  to 
include  review/approval  requirements  of  the 
Service  Acquisition  Strategy  (SAS),  including 
the  first  ever  US D( AT&L)  review  of  SAS 
greater  than  $2B 

2005 

USD  (AT&L)  memorandum,  subject: 
Acquisition  of  Services  Policy  Review 

USD  (AT&L)  requested  a  formal  review  of  the 
acquisition  of  services  to  assess  compliance 
with  Department  pohcy  while  soliciting  views 
on  ways  to  improve  the  pohcy 

2006 

Section  812  of  NDAA  FY  2006  Public 

Law  109-163  (author:  Congress) 

Establishes  the  requirement  for  the  Secretary  of 
Defense  to  establish  and  implement  a 
management  structure  for  the  procurement  of 
contract  services  for  the  DOD.  DOD  is  to  work 
with  the  Military  Components  to  identify  skills 
and  competencies  needed  for  procurement  of 
services  and  develop  strategy  to  recruit  and 
train  employees  for  this 

2006 

USD  (AT&L)  memorandum,  subject: 
Acquisition  of  Services  policy 

Includes  guidelines  for  creating  an  Acquisition 
Strategy,  assigned  duties  for  Senior  Officials 
with  review  and  approval  authorities  for 
acquisitions  of  services  (identified  in  Table  2), 
and  established  procedures  for  the  collection  of 
automated  data  to  be  submitted  to  the 
USD(AT&L) 

2007 

USD  (AT&L)  memorandum,  subject: 
Acquisition  of  Services  Review  and 
Decision  Authority 

Delegated  review  and  decision  authority  to 

DPAP  for  acquisitions  of  services  greater  than 
$1  billion  in  accordance  with  10  U.S.  Code 

2330,  as  amended  by  Section  812  of  the  NDAA 
FY  2006 

2008 

USD  (AT&L)  memorandum,  subject: 

Peer  Reviews  of  Contracts  for  Supplies 
and  Services 

Estabhshed  DOD  policy  for  Peer  Reviews 
Requiring:  pre-award  Peer  Reviews  for  all 
contracts  with  an  estimated  value  of  $1  billion 
or  more  (including  options)  and  post-award 

Peer  Reviews  for  all  service  contracts  with  an 
estimated  value  of  $1  billion  or  more 
(including  options).  Further,  Military 
Departments  will  establish  their  own 
procedures  for  contracts  valued  at  less  than  $1 
billion 
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Table  1  (continued) 


Year 

DOD  Policy/Memorandum/ 
Instruction 

Details 

2008 

Acquisition  of  Services  policy  expressly 
stated  in  Enclosure  9  to  DODl  5000.02 
(author;  Department  of  Defense) 

Incorporates  Acquisition  of  Services  policy 
into  DODl  5000.02.  Provides  detailed 
instructions  on  the  completion  of  OSD  Peer 
Reviews  (pre-award  and  post-award),  and 
establishes  procedures  for  data  collection  and 
assigns  responsibility  for  Military  Departments. 
Revises  the  review  and  approval  authorities  for 
acquisitions  of  services  (identified  in  Table  2) 

2009 

DPAP  Policy  memorandum,  subject; 
Review  Criteria  for  the  Acquisition  of 
Services 

This  policy  provides  the  tenants  and  review 
criteria  for  the  Acquisition  of  Services.  These 
reviews  are  to  ensure  that  the  requirements  are 
clear  and  well  defined,  acquisition  approach 
and  business  strategy  are  appropriate  and  that 
there  are  mechanisms  to  ensure  proper 
oversight  of  contractor  performance 

2012 

DPAP  memorandum,  subject;  Service 
Acquisition  Workshop  (SAW) 

Mandates  SAW  for  contracts  valued  at  >$1B 
unless  waived,  recommends  SAWs  for 
contracts  valued  >$100M.  Requires  the  training 
be  completed  before  a  Service  Acquisition 
Strategy  will  be  approved  by  the  OSD 

2013 

The  Services  Acquisition  Deputy  Directorate  established  within  the  Office  of  DPAP 

2013 

USD  (AT&L)  memorandum,  subject; 
Appointment  of  DOD  Functional  Domain 
Experts  for  Contracted  Services 

Acquisition  Management 

Appointed  senior  DOD  officials  as  Functional 
Domain  Experts  (FDEs)  in  accordance  with  the 
requirements  of  Better  Buying  Power  2.0  to 
oversee  each  Services  Portfolio  Group.  The 
intent  is  to  actively  oversee  the  hfe-cycle 
process  of  services  acquisition  to  include 
forecasting  and  budgeting,  requirements 
definition  and  validation,  procurement,  and 
active  management  and  oversight  of  contracted 
services 

2014 

Performance  of  the  Defense  Acquisition 
System  -  2014  Annual  Report  (author; 
DPAP) 

Services  acquisition  metrics  are  included  in  this 
report  for  the  first  time.  In  terms  of  services, 
the  focus  was  on  incentives-  particularly  from 
contract  types  and  profits  or  fees,  in  addition  to 
updating  the  prior  analysis  where  recent  data 
may  affect  the  statistical  result.  In  conjunction 
with  the  BBP  continuous  improvement  efforts, 
policies  and  structures  related  to  the  acquisition 
of  services  are  being  expanded 

2015 

The  Services  Functional  Integrated  Product  Team  (IPT)  Charter  was  updated  to  reflect  the 
appointment  of  the  Deputy  Director,  DPAP/  SA  as  the  Functional  Fead  for  Services 

2016 

DODl  5000.74  Services  Acquisition  was 
released  and  DODl  5000.02,  Enclosure  9 
is  canceled  (author;  Department  of 

Defense) 

Addressed  in  detail  below 
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Table  2  describes  the  changes  in  policy  requirements  throughout  the  past  decade. 
While  there  have  been  many  modifications  to  the  structure  and  layout  of  the  approval 
authorities,  the  thresholds  have  not  been  significantly  altered.  Prior  to  the  release  of 
Department  of  Defense  Instruction  (DODI)  5000.74,  the  USD(AT&L)  was  responsible  to 
approve  the  service  acquisition  strategy  for  all  actions  meeting  or  exceeding  $1B 
(Headquarters,  Department  of  the  Army,  2010),  whereas  the  latest  guidance  provides  the 
USD(AT&L)  the  authority  to  designate  an  OSD  individual  or  the  Military  Department 
Service  Acquisition  Executive  (SAE)  to  further  reduce  bureaucracy  and  delays 
(Department  of  Defense,  2016). 


Table  2.  Comparison  of  Approval  Authorities  for  the  Acquisition  of  Services 


Comparison  ol 

'  Approval  Authorities  for  the  Acguisition  of  Services 

Category 

Acquisition  of 
Services  Policy 
(2006) 

DODI  5000.02 
Enclosure  9  (2008) 

DODI  5000.74  (2016) 

Services  Category  I 

Estimated  Value: 

$250  million  or 
more  *Note  that 
estimated  value 
>$1B  shall  be 
referred  to 
USD(AT&L) 

Decision  Authority; 

Senior  Official  or  as 
designated 

Estimated  Value:  $250 
million  or  more 

Decision  Authority: 

Senior  Official  or  as 
designated 

Estimated  Value:  Greater 
than  $  1  billion  or  more  than 
$300  million  in  any  one 
year 

Decision  Authority: 

USD(AT&L)  or  designee, 
or  Military  Department 
Service  Acquisition 

Executive  (SAE)  or 
designee 

Acquisitions  >$1B 

N/A 

Estimated  Value: 

Greater  than  $  1  billion 

Decision  Authority; 
USD(AT&L)  or  designee 

N/A 

Special  Interest 

Estimated  Value: 

As  designated  by 
USD(AT&L)  or 
other  Senior  Official 

Decision  Authority: 

USD(AT&L)  or 
Senior  Officials 

Estimated  Value:  As 

designated  by 

USD(AT&L)  or  Military 
Department  Senior 

Official 

Decision  Authority: 

USD(AT&L)  or  Senior 
Officials 

Estimated  Value:  As 

designated  by  USD(AT&L) 

Decision  Authority: 

USD(AT&L)  or  designee 
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Table  2  (continued) 


Category 

Acquisition  of 
Services  Policy 
(2006) 

DODI  5000.02 
Enclosure  9  (2008) 

DODI  5000.74  (2016) 

Estimated  Value; 

$10  million  but  less 
than  $250  million 

Estimated  Value:  $10 

million  but  less  than  $250 
million 

Estimated  Value:  $250 
million  or  more,  but  less 
than  $1  billion 

Services  Category  II 

Decision 

Authority:  Senior 
Official  or  as 
designated 

Decision  Authority; 

Senior  Official  or  as 
designated 

Decision  Authority: 

USD(AT&L)  or  designee, 
or  Military  Department 

SAE  or  designee 

Services  Category 

III 

Estimated  Value: 

Greater  than 
Simphfied 
Acquisition 
Thi-eshold  (SAT) 
but  less  than  $10 
million 

Decision 

Authority:  Senior 
Official  or  as 
designated 

Estimated  Value: 

Greater  than  SAT  but  less 
than  $10  million 

Decision  Authority: 

Senior  Official  or  as 
designated 

Estimated  Value:  $  1 00 

million  or  more,  but  less 
than  $250  million 

Decision  Authority: 

Senior  Services  Manager 
(SSM)  or  designee 

Services  Category 

IV 

N/A 

N/A 

Estimated  Value:  $10 

milhon  to  less  than  $100 
million 

Decision  Authority:  SSM 

or  designee 

Services  Category  V 

N/A 

N/A 

Estimated  Value;  Greater 
than  SAT  but  less  than  $10 
million 

Decision  Authority:  SSM 

or  designee 

1.  2016:  Department  of  Defense  Instruction  (DODI)  5000.74,  Subject: 

Defense  Acquisition  of  Services,  05  January  2016 

•  Cancels  DODI  5000.02  Enclosure  9,  Acquisition  of  Services  policy 

•  Procedures  in  this  instruction  apply  to  the  entire  DOD,  however,  DOD 
Component  decision  authorities  may  tailor  to  best  achieve  cost,  schedule  and 
performance  objectives 
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•  The  DOD  will  manage  services  acquisitions  in  order  to  improve 
productivity  and  efficiency 

•  DOD  Component  heads  implement  the  policy  and  procedures  in  this 
instruction  within  their  respective  Component 

•  Portfolio  management  enables  a  framework  for  strategic  oversight  by  the  USD 
(AT&L),  coupled  with  decentralized  execution  by  the  DOD  Components 

o  Requires  collection  and  sharing  of  data  within  and  across 

organizations,  employing  cross-departmental  teams,  and  identification 
and  promulgation  of  best  practices  and  lessons  learned 

o  Collaboration  amongst  requiring  organizations  and  contracting 

activities  enables  the  DOD  to  leverage  its  resources  and  buying  power. 
By  fostering  collaboration,  portfolio  management  can:  improve 
requirements  transparency  across  DOD,  reduce  redundancy  of  similar 
service  requirements  and  allow  for  increased  awareness  of  alternative 
solutions  to  better  meet  the  needs  of  the  requiring  activity. 

o  Using  the  DOD  portfolio  taxonomy  (portfolio  groups,  see  Figure  2) 
ensures  they  are  overseen  through  the  Functional  Domain  Expert 
(FDE)  structure  (see  Eigure  3) 

•  Establishes  leaders  through  the  USD(AT&E)  and  DOD  Components  and  their 
duties  (see  Table  3) 


Transportation 

Services 


Logistics 

Management 

Services 


Equipment 
Related  Services 


Electronics  & 
Comm.  Services 


Medical 

Services 


Facilities 
Related  Services 


Knowledge 
Based  Services 


Research  and 
Development 


Construction 

Services 


Eigure  2.  Services  Acquisition  Portfolio  Groups. 
Source:  Department  of  Defense  (2016). 
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USD(AT&L)  or  Designee 


Executive  Secretariat,  DPAP 


Component-Level  Leads 


Army 


Air 

Force 


Navy 


Defense 

Loeistics 

Agency 


Missile 

Defense 

Agency 


Defense 

Health 

Agency 


DPAP  &  SSMs  Provide  Cross 
Cutting  Staff  Support 


Transportation 
Services  FDE 

Logistics 
Management 
Services  FDE 

Equipment 

Related 

Services  FDE 

Electronics  & 

Comm.  Services 

FDE 

Medical 

Services  FDE 

Facilities 

Related 

Services  FDE 

Knowledge 
Based  Services 
(FDEs  assigned  by 
porrfoiio) 

Component 
Level  Leads 

Component 
Level  Leads 

Component 
Level  Leads 

Component 
Level  Leads 

Component 
Level  Leads 

Component 
Level  Leads 

Component 
Level  Leads 

Engineering  & 
Technical  Services 
FDE 


CLLs 


Program  Mgmt 
Services  FDE 


CLLs 


Management 
Support  Services  FDE 


CLLs 


Administrative  & 
Other  Services  FDE 


CLLs 


Professional 
Services  FDE 


CLLs 


Education  &  Training 
FDE 


CLLs 


Figure  3.  Functional  Domain  Expert  Structure.  Source:  Department  of  Defense  (2016). 
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Table  3.  Personnel  Requirements  due  to  DODI  5000.74.  Source:  Department  of 

Defense  (2016) 


Personnel  Requirements  due  to  DODI  5000.74 

Appointed  by  USD(AT&L) 

Functional  Domain  Expert 
(FDE) 

DOD-level  lead  reporting  through  Component  head  to  USD 
(AT&L)  for  their  service  portfolio  group  to  provide  effective 
strategic  oversight  of  contract  service  acquisitions.  Will  be 
designated  for  each  of  the  service  portfolio  groups  and  collaborates 
to  define  common  processes  across  DOD 

Component  Level  Lead  (CEL) 

Aligned  within  FDE  structure  to  support  the  strategic  management 
and  leadership  portfolio  groups  across  the  DOD  Components  to 
improve  acquisition  of  services 

Portfolio  Specific  Commodity 
Managers  (PSCMs) 

Support  efforts  of  the  Senior  Services  Managers  and  requiring 
activities 

Appointed  by  DOD  Component 

Service  Acquisition  Executive 
(SAE) 

Senior  official  responsible  for  management  of  acquisition  of 
contracted  services,  further  delegation  to  Senior  Procurement 
Executive  (SPE) 

Senior  Services  Manager  (SSM) 

Service  acquisition  experts  and  decision  authorities  under  the 
Secretaries  of  Military  Department  responsible  for  planning, 
strategic  sourcing,  execution  and  management  of  acquisition  of 
contracted  services.  Establish  appropriate  management  structures 
and  processes  to  ensure  effective  implementation  and  execution  of 
DODI  5000.74  and  identify,  forecast  and  track  pending 
requirements 

•  Director,  Defense  Procurement  and  Acquisition  Policy  (DPAP) 
coordinates  (as  appropriate)  with  the  FDE  regarding  specific  portfolio  or 
portfolio  group  for  the  acquisition,  and  with  the  OSD  Office  of  Small 
Business  Programs  (OSBP)  on  all  proposed  acquisition  of  services 
acquisition  strategies 

•  Services  Requirements  Review  Board  (SRRB)  process  will  be  used  for 
services  acquisitions 

o  At  or  above  $10M  annually  (excluding  options  that  were  previously 
considered  and  approved)  to  review,  validate,  prioritize  and  approve 
services  requirements  to  accurately  inform  the  budget  and  acquisition 
processes 

o  Requirements  approval  should  be  obtained  from  the  assigned  SRRB 
chair  before  the  initiation  of  any  acquisition  action  unless  otherwise 
directed  by  the  decision  authority 
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o  SRRB  must  be  validated  before  the  approval  of  the  acquisition 
strategy.  The  SRRB  will: 

a.  Increase  visibility  of,  and  collaboration  on,  services 
requirements  among  stakeholders 

b.  Validate  requirements  before  a  contract  is  awarded 

c.  Provide  for  prioritization  of  services  requirements  to  support 
funding  decisions 

d.  Increase  collaboration  among  stakeholders  on  key  strategy 
decisions  to  optimize  services  acquisitions  and  enable 
efficiencies 

e.  Foster  proactive  management  by  the  Components  for  services 
acquisitions 

f.  Will  provide  a  process  for  assessing,  reviewing  and  validating 
services  requirements  by  senior  leaders  to  include  mission 
need,  workforce  analysis,  strategic  alignment,  relationship  to 
other  requirements,  prioritization,  and  market  research. 
Approval  of  the  requirements  should  be  obtained  from  the 
SRRB  chair  before  any  acquisition  action  is  initiated  and 
documented  in  the  Acquisition  Plan 

•  Multi-Functional  Teams  supporting  services  contract  requirements  valued 
at  $1B  or  more  will  participate  in  a  SAW  before  seeking  acquisition 
strategy  approval 

o  A  SAW  is  a  Defense  Acquisition  University  facilitated  course 
where  the  instructors  mentor  the  team  and  guide  in  developing  their 
market  research,  performance  requirements,  acquisition  planning, 
source  selection  process  and  contractor  performance  assessment 
planning  and  execution  documents 

o  The  team  learns  the  Seven-Step  Service  Acquisition  Process,  as 
well  as  how  to  establish  performance  based  requirements  in  the 
Acquisition  Requirements  Roadmap  Tool  (ARRT) 

•  Submission  of  the  Acquisition  Strategies  to  OSD  for  review  and  approval 
are  required: 

o  Before  the  final  solicitation  is  issued 

o  To  submit  to  the  Director,  DPAP  for  any  proposed  acquisition  of 
services  for  which  the  USD(AT&L)  or  designee  is  the  decision 
authority 
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o  Will  review  within  30  days  after  receipt.  Criteria  and  tenants  are 
addressed  in  the  Defense  Federal  Acquisition  Regulation 
Supplement  (DFARS)  237.102-76 

o  After  review,  the  result  will  be  either  approval  or  disapproval  of  the 
acquisition  strategy 

o  Review  and  approval  process  will  precede  the  peer  review  of  the 
request  for  proposal 

•  The  Director,  DPAP  will  conduct  peer  reviews  for  all  service  acquisitions 
with  an  estimated  value  of  $1B  or  more  or  $500M  for  non-competitive 
acquisitions  (DFARS  201.170) 

•  FDEs  and  their  leadership  teams  will  provide  the  USD  (AT&L)  semi¬ 
annual  reports.  The  updates  will  review  implementation  of  the  strategic 
management  processes  and  framework  to  lead,  manage  and  support  the 
effective  and  efficient  acquisition  of  services.  (Department  of  Defense, 
2016) 

2.  Other  Notable  Considerations 

DFARS  Procedures,  Guidance  and  Information  (PGI)  237.102-77  includes  the 
requirements  for  the  Automated  Requirements  Roadmap  Tool  (ARRT),  which  was 
created  to  assist  requiring  activities  to  develop  their  performance-based  requirements  by 
providing  a  standard  template  and  some  default  text  that  can  be  modified  to  fit  each 
requirement.  The  DFARS  PGI  further  states  “the  tool  should  be  used  to  prepare  contract 
documents  for  all  performance-based  acquisitions  for  services.”  Acquisitions 
participating  in  the  SAW  will  be  taught  how  to  use  the  ARRT.  (DFARS  PGI  237.102- 
77).  Additionally,  acquisitions  meeting  the  threshold  for  OSD  Peer  Review  will  be  asked 
to  justify  if  they  did  not  use  ARRT  to  develop  their  requirements  as  it  is  a  best  practice. 

DPAP  has  created  a  Services  Acquisition  website  to  provide  information 
including  policies  and  guidance,  information  on  the  assigned  FDEs  and  SSMs,  SRRB, 
and  training.  Additionally,  the  website  provides  information  for  both  pre  and  post-award 
Peer  Reviews  to  include  frequently  asked  questions.  Peer  Request  forms,  questions  asked 
for  the  Peer  Reviews  and  most  importantly,  best  practices.  This  website  helps  facilitate 
information  sharing  by  DPAP  and  can  assist  to  ensure  that  submissions  to  DPAP  are 
more  consistent  (DPAP  Service  Acquisition,  n.d.) 
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B.  THE  UNITED  STATES  ARMY 

The  Army  is  required  to  implement  the  OSD  Service  Acquisition  requirements, 
but  also  has  the  ability  to  add  supplemental  requirements  as  they  deemed  fit.  This  section 
will  address  the  supplemental  requirements  for  service  acquisitions  through  the  Army. 

As  stated  in  a  briefing  by  the  SSM,  the  Army  has  added  Governance  and 
Oversight  through:  Service  Acquisition  Guidance  (Policy),  Portfolio  Management  (by 
establishing  a  Portfolio  Coordinator/Manager  structure).  Strategic  Sourcing,  Acquisition 
Strategy  Approval/Peer  Reviews,  SAWs,  Service  Acquisition  Training  (to  non¬ 
acquisition  workforce).  Audits,  and  Service  Requirements  Reviews.  Assistant  Secretary 
of  the  Army  for  (Acquisition,  Logistics  and  Technology)  ASA(ALT)  Services 
Acquisition  Update  briefing.  May  12,  2015  (W.  Mercer,  personal  communication,  August 
18,2015).! 

Policies  that  lead  the  Army:  DODI  5000.74,  Army  Federal  Acquisition 
Regulation  Supplement  (AFARS)  5137-Service  Contracting,  Army  Regulation  (AR)  70- 
13  Management  and  Oversight  of  Service  Acquisitions,  and  the  Army  Services 
Implementation  Plan-Optimization  of  Army  Services  Acquisition.  Although  DODI 
5000.74  is  the  leading  policy,  as  of  the  conception  of  this  JAP,  the  Army  has  not  released 
updated  policy  in  accordance  with  the  changes.  The  provided  Army  policies  do  not 
include  the  latest  requirements/updates  and  therefor,  have  not  been  included  in  this 
section,  unless  interim  guidance  was  provided  through  points  of  contact,  as  specified 
further  throughout  this  section. 

Below  is  a  brief  explanation  of  each  Army  policy,  in  order  of  timeframe  released 
and  their  supplemental  requirements  enacted  for  service  acquisitions. 


!  A  PowerPoint  briefing  to  the  AS  A(ALT)  was  provided  via  email  by  a  portfolio  coordinator  at  the 
Office  of  the  DASA(P)  SSM. 
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1.  2010  Army  Regulation  (AR)  70-13  Management  and  Oversight  of 
Service  Acquisitions 

•  Provides  pre-award  requirements  and  documents  required  for  Army 
service  acquisitions  from  advance  planning  and  service  contract  approval 
through  contract  formation 

•  Upon  the  release  of  the  DODI  5000.74,  the  Army  is  in  the  process  of 
updating  this  regulation  and  will  provide  the  revision  once  complete 

•  Established  the  requirement  for  the  Headquarters  Department  of  the  Army 
(HQDA)  Army  Services  Strategy  Panel  (ASSP)  for  service  acquisitions 
that  require  USD  (AT&L)  review  and  approval.  The  purpose  of  the  ASSP 
is  to  “provide  senior  Army  functional  principals  the  opportunity  to  review 
proposed  acquisition  strategies  and  metrics  for  service  acquisitions  and 
reach  consensus  on  the  strategies  that  are  most  advantageous  to  the 
Army.”  (Headquarters,  Department  of  the  Army,  2010,  p.  10) 

•  Strategic  sourcing  is  the  primary  benefit  to  advance  acquisition  planning 
to  ensure  the  most  efficient  and  advantageous  manner  is  utilized  by 
leveraging  amongst  organizations  and  to  manage  procurement  spending 
while  seeking  better  terms  and  conditions  for  the  life  of  the  contract. 
(Headquarters,  Department  of  the  Army,  2010) 

2.  2011  Army  Services  Implementation  Plan-Optimization  of  Army 
Services  Acquisition 

Prior  to  the  release  of  the  Optimization  of  Army  Services  Acquisition  in  201 1,  the 
Army  did  not  have  standardization  for  services  acquisitions.  Commands  and  activities 
needed  the  processes  and  tools  to  provide  visibility  in  what  is  being  procured.  The 
Implementation  Plan  describes  what  the  Army  requests  the  organizational  structure  and 
integrated  processes  to  be,  states  the  Commander  has  oversight  and  accountability  for 
their  agency’s  services  spending  and  to  ensure  the  necessary  visibility  of  services 
acquisitions  is  available. 

•  The  plan  establishes  the  portfolio  management  approach  to  establish  an 
oversight  and  management  process  that  will  identify,  track  and  monitor 
projected  savings.  The  Army  SSM  was  formed,  as  required  by  OSD 
policy.  (Office  of  the  Assistant  Secretary  of  the  Army,  2011) 


17 


3. 


AFARS  5137-  Service  Contracting 


The  AFARS  was  last  updated  on  25  August  2015,  and  does  not  yet  include  the 
requirements  changes  since  the  release  of  DODI  5000.74.  The  Service  Contracting  part 
provides  the  requirements  in  addition  to  the  Federal  Acquisition  Regulation  and  DFARS, 
to  include  reference  to  Army  Regulation  (AR)  70-13,  Army  Service  Acquisition  Strategy 
(SAS)  review  requirements,  and  most  important  to  this  topic,  requirements  for  the  ASSP 
and  SAS  content. 

The  ASSP  process  was  initiated  upon  the  release  of  AR  70-13.  Formal  guidance 
was  provided  in  AFARS,  which  is  balanced  with  the  approval  authorities  of  the  SAS  as 
described  in  Table  4. 


Table  4.  AFARS  Approval  Guidance  of  SAS 


AFARS  Approval  Guidance  of  SAS- 
based  on  DODI  5000.02  Enel  9  guidance 

ASSP  Review  Level 

Acquisitions  >$1B 

Estimated  Value:  Greater  than  $1  billion 

Decision  Authority:  USD(AT&L) 

HQDA  level,  following  an  OSD 

Peer  Review 

Special  Interest 

Estimated  Value:  As  designated  by 
USD(AT&L)  or  Military  Department  Senior 
Official 

Decision  Authority:  USD(AT&L) 

HQDA  level 

Services  Category  I 

Estimated  Value:  $250  million,  but  less  than 
$1  billion 

Decision  Authority:  DASA(P)  or  SSM 

HQDA  level 

Services  Category  II 

Estimated  Value:  $50  million  but  less  than 
$250  million 

Decision  Authority:  Head  of  Contracting 
Activity  (HCA)  (delegable  no  lower  than 
Principal  Assistant  Responsible  for 

Contracting  (PARC),  Program  Executive 

Officer  (PEO)  or  direct  reporting  Program 
Manager  (PM) 

PEO,  direct  reporting  PMs  and 

HCA  level  (delegable  to  PARC) 

Services  Category  III 

Estimated  Value:  Greater  than  SAT  but  less 
than  $50  million 

Decision  Authority:  HCA  (delegable  no 
lower  than  the  CCO)  or  PM 

HCA  level  (delegable  to  PARC) 
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The  ASSP  concludes  upon  receiving  approval  of  the  SAS.  The  final  solicitation 
cannot  be  posted  until  the  SAS  has  been  approved.  For  service  acquisitions  requiring 
HQDA  review  and  approval  (over  $250  M,  as  reflected  in  Table  4),  this  process  can  take 
between  30  and  60  days  (AFARS  5137(f)). 

The  Implementation  Plan  established  the  portfolio  management  approach  for 
oversight,  which  ultimately  led  to  the  creation  of  the  Portfolio  Coordinators  and  Portfolio 
Managers  to  provide  support  to  the  Army  services  acquisition  oversight  by  conducting 
spend  and  trend  analysis;  facilitating  collaboration  between  requiring  activities;  and 
identifying  potential  strategic  sourcing  opportunities.  Portfolio  Coordinators  are  SSM 
staff,  while  Portfolio  Managers  are  Command  Level  staff,  with  one  manager  aligned  to 
each  of  the  DOD  portfolio  taxonomies  (Office  of  the  Assistant  Secretary  of  the  Army, 
2011). 


4.  Other  Notable  Considerations 

A  DPAP  Memorandum  established  the  requirement  to  participate  in  a  SAW  for 
service  acquisitions  meeting  or  exceeding  $1B  (Director,  DPAP,  2012);  however,  to 
further  establish  the  Army  requirements,  DASA(P)  SSM  lowered  the  threshold  to 
$250M+,  and  strongly  recommended  for  service  acquisitions  $10M  (Office  of  the 
Assistant  Secretary  of  the  Army,  Acquisition,  Logistics  and  Technology,  2013). 

C.  GOVERNMENT  ACCOUNTABILITY  OFFICE  (GAO) 

As  the  federal  government’s  largest  buyer  of  services  provided  by  contractors 
(Cooper,  2005),  DOD’s  obligations  for  services  in  2014  totaled  $156  billion,  accounting 
for  more  than  half  of  total  contract  spending  (DiNapoli,  2016).  Although  DOD  has  taken 
several  actions  to  make  improvements,  it  still  continues  to  face  challenges  related  to  the 
management  of  service  acquisitions. 

This  section  will  review  those  challenges  and  the  events  leading  up  to  the  current 
policies  and  directives  and  subsequently,  the  establishment  of  the  additional  Army 
service  acquisition  reviews. 

In  2002,  William  T.  Woods  testified  that  DOD  spending  was  not  being  managed 
efficiently  and  that  “millions  of  service  contract  dollars  were  at  risk  at  defense  and 
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civilian  agencies  because  acquisitions  were  poorly  planned,  not  adequately  competed,  or 
poorly  managed”  (Woods  2002,  p.  1).  To  help  guide  DOD’s  efforts,  GAO  examined 
strategic  frameworks  that  industry  adopted  in  order  to  find  ways  to  leverage  their 
resources  to  better  manage  their  relationships  with  their  suppliers  and  control  costs.  In 
2001,  “the  Secretary  of  Defense  noted  that  DOD  is  working  to  adapt  the  same 
‘revolutionary  business  and  management  practices  that  helped  the  commercial  sector  gain 
a  competitive  edge  in  a  rapidly  changing  global  marketplace’”  (Brock,  2002,  p.  17).  In 
the  same  year,  GAO  reported  that  the  DOD  operated  like  companies  before  they  adopted 
a  strategic  approach  where  responsibilities  for  acquiring  services  were  divided  amongst 
the  different  commands  making  it  difficult  to  coordinate  service  contracting  initiatives.  In 
response,  “Army  officials  noted  that  they  are  evaluating  how  to  centralize  the  processes 
for  acquiring  services  within  the  Office  of  the  Secretary  of  the  Army”  (Brock,  2002,  p. 
20). 

According  to  GAO  report.  Defense  Acquisitions:  Tailored  Approach  Needed  to 
Improve  Service  Acquisition  Outcomes,  in  May  of  2002,  DOD  established  policy 
requiring  the  development  of  a  review  process  and  oversight  thresholds  for  individual 
service  acquisitions.  Thereafter,  military  departments  were  tasked  with  establishing 
similar  review  processes.  “The  Air  Force,  Army,  and  Navy  each  developed  individual 
service  acquisition  review  processes  and  authorities  to  support  the  DOD  review 
requirements  and  identified  respective  decision  authorities  responsible  for  conducting 
execution  reviews  to  assess  progress  against  metrics”  (Francis,  2006,  p.  23).  Also 
significant  in  2002,  was  the  formation  of  the  Services  Acquisition  Reform  Act  of  2002. 
One  main  goal  of  the  bill  was  to  improve  the  management  of  service  acquisitions  (Woods 
2002). 

In  2003,  GAO  report  Contract  Management:  High-Level  Attention  Needed  to 
Transform  DOD  Services  Acquisition  reported  that  “recent  legislation  directs  DOD  to 
manage  its  services  procurement  more  effectively”  (Cooper,  2003,  Highlights).  However, 
DOD’s  management  reviews  for  services  acquisitions  were  not  a  good  representation  of 
their  strategy  to  be  more  effective  for  DOD  and  the  military  departments.  Cooper  stated, 
“Greater  attention  is  needed  by  DOD  management  to  promote  a  strategic  orientation  by 
setting  performance  goals,  including  savings  goals,  and  ensuring  accountability  for 
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achieving  them”  (Cooper,  2003,  “Highlights”).  Continuing  their  efforts  in  2002,  Cooper 
reported  that  using  a  strategic  approach  modeled  after  industry  could  help  reform  DOD. 

Congress  included  provisions  in  the  National  Defense  Authorization  Act 
for  Fiscal  Year  2002  to  achieve  significant  savings  through  improved 
management  and  oversight  of  services  procurement.  Specifically,  section 
801  of  this  law  requires  DOD  to  establish  (1)  a  management  structure 
designed  to  provide  visibility  and  establish  accountability  for  services 
contracts,  (2)  a  program  review  structure  for  major  services  acquisitions, 
and  (3)  an  automated  system  to  collect  and  analyze  data  to  support 
management  decisions  in  contracting  for  services.  (Cooper,  2003,  p.  2) 

In  2001,  the  Army  was  given  approval  to  consolidate  contracting  activities  which 
established  the  Army  Contracting  Agency  in  October  2002  in  an  effort  to  reduce  costs. 
“The  agency’s  organizational  structure  assigns  regional  executive  responsibility  for 
managing  services  contracting,  and  included  a  high-level  council  in  headquarters  for 
overseeing  more  strategic  approaches  to  buying  Army  installation  support  services.” 
(Cooper,  2003,  p.  12) 

In  a  2005  GAO  report.  Contract  Management:  Opportunities  to  Improve 
Surveillance  on  Department  of  Defense  Service  Contracts,  Cooper  reported  that  “the 
Army  does  not  require  surveillance  personnel  to  be  assigned  responsibility  prior  to 
contract  award”  (Cooper,  2005,  Highlights).  Furthermore  because  surveillance  duties 
were  considered  a  part-time  responsibility,  they  were  not  evaluated  on  their  surveillance 
performance  (Cooper,  2005).  Despite  efforts  to  implement  the  provisions  in  the  FY2002 
NDAA,  there  was  little  improvement  in  service  contract  surveillance  (Cooper,  2005).  In 
this  same  report,  it  was  found  that  DOD  was  unable  to  communicate  their  plan  to 
improve  the  acquisition  of  services,  identify  necessary  changes  and  processes  to  prioritize 
risks  or  even  understand  service  spending  trends.  At  the  time  of  this  report,  DOD’s 
current  strategy  did  not  address  how  they  intended  to  manage  risk  or  integrate  key  service 
acquisition  initiatives  (Cooper,  2005). 

In  2006,  “GAO  identified  DOD  contract  management  to  be  at  high  risk  of 
vulnerabilities  to  fraud,  waste,  abuse,  and  mismanagement  for  more  than  a  decade” 
(Francis,  2006,  p.  1).  Some  reasons  for  vulnerabilities  were  identified  by  GAO’s  2005 
high-risk  report  and  included  a  lack  of  a  strategic  approach  to  buying  services  and  a 
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comprehensive  plan  to  assess  skills  and  capabilities  in  the  acquisition  workforce. 
Additionally,  DOD  was  not  fully  considering  all  available  tools  and  resources  to  acquire 
services  (Francis,  2006).  “DOD’s  current  approach  to  managing  service  acquisition  has 
tended  to  be  reactive  and  has  not  fully  addressed  the  key  factors  for  success  at  either  the 
strategic  or  the  transactional  level”  (Francis,  2006,  p.  3). 

In  April  2009  GAO  report.  Defense  Acquisitions:  Actions  Needed  to  Ensure  Value 
for  Service  Contracts,  John  Hutton  and  William  Solis  stated  that  “DOD  does  not  always 
use  sound  practices  when  acquiring  services  and  the  department  lacks  sufficient  people 
with  the  right  skills  to  support  its  acquisitions”  (Hutton  &  Solis,  2009,  Highlights). 
Hutton  and  Solis  reported  that  a  review  of  Army  services  contracts  indicated  that  contract 
oversight  was  inadequate  due  to  lack  of  trained  personnel.  Additionally,  due  to 
incomplete  files  and  lack  of  proper  documentation,  contract  oversight  for  services 
contracts  supporting  contingency  operations  was  inadequate  (Hutton  &  Solis,  2009). 

In  September  2010,  USD(AT&L)  issued  the  first  of  three  BBP  memoranda  that 
included  steps  to  improve  service  acquisitions.  In  the  November  2010  memorandum  that 
followed  it,  USD(AT&L)  directed  that  each  military  department  establish  the  position  of 
a  SSM,  responsible  for  planning,  executing,  strategically  sourcing,  and  managing  service 
contracts  for  their  military  departments,  and  to  serve  as  the  approval  authority  for  service 
acquisitions  valued  at  less  than  $250M  (DiNapoli,  2016). 

In  September  2011,  Belva  M.  Martin  reported,  “according  to  the  Defense  Science 
Board,  buying  services  was  fundamentally  different  than  buying  weapon  systems  because 
of  the  time  it  generally  takes  to  move  from  the  identification  of  a  requirement  to  contract” 
(Martin  2011,  p.  1).  In  GAO  report.  Defense  Acquisitions:  Goals  and  Associated  Metrics 
Needed  to  Assess  Progress  in  Improving  Service  Acquisition,  DiNapoli  reported  that,  of 
the  contracts  reviewed,  more  than  half  of  the  personnel  with  acquisition  related 
responsibilities  were  non-acquisition  personnel  (DiNapoli,  2013).  Over  the  past  decade, 
the  same  issues  have  been  raised  about  the  challenges  that  DOD  faces  with  services 
acquisitions. 

In  2012,  in  an  effort  to  assist  DOD  personnel  develop  better  acquisitions, 
USD(AT&L)  collaborated  with  DAU  to  develop  the  Acquisition  Requirements  Roadmap 
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Tool  (ARRT).  GAO  reported,  although  it  was  not  mandated  to  be  used  across  DOD,  the 
feedback  was  positive.  Its  intended  purpose  of  helping  personnel  write  critical  documents 
such  as  performance  based  requirements  and  performance  work  statements,  was  apparent 
in  these  documents  better  reflecting  the  requirements.  In  this  same  year,  USD(AT&L) 
mandated  the  use  of  the  SAW.  Additionally,  “USD(AT&L)  established  the  Acquisition 
of  Services  Functional  Integrated  Product  Team  (Services  FIPT)  in  August  2012,  in  part, 
to  address  training  requirements  in  10  U.S.C.  §  2330  acquisitions”  (DiNapoli,  2013,  p.  I 
p.  II).  One  of  the  tasks  for  Services  FIPT’s  was  to  identify  how  to  properly  train  non¬ 
acquisition  personnel  involved  in  service  acquisitions  (DiNapoli,  2013).  Unfortunately,  at 
the  time  of  report,  the  Services  FIPT  had  made  little  progress. 

GAO  reported  in  June  2013  that,  while  DOD  took  actions  to  address  legislative 
requirements,  they  were  not  positioned  to  determine  its  impacts  to  services  acquisition.  In 
part,  DOD  was  unable  to  define  a  desired  end  state  because  the  available  data  was  not 
being  utilized  to  determine  the  “current  status  of  service  acquisition  in  terms  of  the 
volume,  type,  location,  and  trends”  (Di  Napoli  2013,  Highlights).  Additionally,  because 
of  DOD’s  inability  to  develop  a  strategy  to  utilize  spending  data  for  informed  decision 
making,  DOD’s  contract  management  remained  on  GAO’s  High  Risk  List  (DiNapoli, 
2013). 

Still  on  the  2015  High  Risk  Series,  GAO  reported  that  DOD  lacked  an  action  plan 
to  guide  its  efforts  in  service  acquisitions.  And  although  DOD  acknowledged  the  need  for 
a  strategy  for  acquiring  services,  they  still  lacked  one,  as  well  as  the  necessary  reliable 
data. 

DOD  has  taken  several  important  steps  to  improve  its  management  of  services  as 
well  as  to  increase  its  knowledge  of  and  visibility  into  service  contract  spending.  Until 
DOD  changes  its  approach  to  comply  with  reporting  requirements  and  captures  all 
contracted  services  in  its  budget  exhibits.  Congress  will  not  have  the  statutorily  required 
information  needed  to  help  conduct  oversight  (DiNapoli,  2016). 
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D.  BETTER  BUYING  POWER 


Another  factor  that  influences  the  ASSP  review  processes  is  Better  Buying  Power 
(BBP),  which  is  a  series  of  iterative  acquisition  reform  initiatives  released  by  the 
Secretary  of  Defense  Acquisition,  Technology  and  Logistics. 

BBP  is  the  implementation  of  best  practices  to  strengthen  the  Defense 
Department’s  buying  power,  improve  industry  productivity,  and  provide 
an  affordable,  value-added  military  capability  to  the 
Warfighter.  Launched  in  2010,  BBP  encompasses  a  set  of  fundamental 
acquisition  principles  to  achieve  greater  efficiencies  through  affordability, 
cost  control,  elimination  of  unproductive  processes  and  bureaucracy,  and 
promotion  of  competition.  BBP  initiatives  also  incentivize  productivity 
and  innovation  in  industry  and  Government,  and  improve  tradecraft  in  the 
acquisition  of  service.  (Department  of  Defense,  n.d.) 


A  summary  of  the  BBP  release  timeline  is  provided  in  Figure  4. 


Figure  4.  BBP  Release  Timeline 


Since  the  launch  of  the  initiative,  there  have  been  three  series  of  BBP.  BBP  3.0  is 
the  latest  iteration  released  in  December  2014  continued  to  build  upon  seven  focus  areas 
established  during  BBP  2.0.  The  latest  version  also  provides  further  direction  by  creating 
initiatives  under  each  focus  area.  Applicable  information  outlined  in  Table  5. 
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Table  5.  BBP  Influences  on  Army  Service  Acquisition  Process.  Source: 

Department  of  Defense  (n.d.) 


BBP  Influences  on  Army  Service  Acquisition  Process 

Applicable  BBP  Focus  Areas 

Description 

Applicable  Initiatives 

Achieve  Affordable  Programs 

Conducting  a  program  at  a 
cost  constrained  by  the 
maximum  resources  the 
Department  can  allocate  for  a 
capability.  These  resources 
include  funding,  schedule  and 
manpower 

•  Mandate  affordability  as  a 
requirement 

Eliminate  Unproductive 
Processes  and  Bureaucracy 

Unnecessary  and  low-value 
added  processes  and  document 
requirements  are  a  significant 
drag  on  acquisition 
productivity  and  must  be 
aggressively  identified  and 
eliminated. 

•  Reduce  frequency  of  OSD 
level  reviews 

•  Re-emphasize  AE,  PEO 
and  PM  responsibility  and 
accountability 

•  Reduce  cycle  times  while 
ensuring  sound  investment 
decisions 

Promote  Effective 

Competition 

Real  competition  is  the  single 
most  powerful  tool  available 
to  the  Department  to  drive 
productivity. 

•  Emphasize  competition 
strategies  and  creating  and 
maintaining  a  competitive 
environment 

•  Increase  small  business 
roles  and  opportunities 

Improve  Tradecraft  in 
Acquisition  of  Services 

The  substantial  amount  of 
money  spent  on  contract 
support  services  demands  a 
management  structure  to 
strategically  source  these 
goods  and  services. 

•  Assign  senior  managers 
for  acquisition  of  services 

•  Improve  requirements 
definition  /  prevent 
requirements  creep 

•  Increase  small  business 
participation 

•  Strengthen  contract 
management  outside  of 
normal  acquisition  chain 

•  Expand  use  of 
requirements  review 
boards  and  tripwires 

Improve  Professionalism  of 
Total  Acquisition  Workforce 

It  is  the  duty  of  the  acquisition 
workforce  to  conduct  itself 
with  excellence, 
responsibility,  integrity  and 
accountability. 

•  Establish  a  higher 
standard  for  key 
leadership  positions 

•  Establish  stronger 
professional  qualification 
requirements  for  all 
acquisition  specialties 

•  Continue  to  increase  the 
cost  consciousness  of  the 
acquisition  workforce 
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III.  POLICY  IMPACT  ANALYSIS 


The  Army  established  goals  in  accordance  with  the  Better  Buying  Power  tenant, 
Improvements  in  Services  Tradecraft.  The  objective  was  to  produce  $2.5  billion  in  Army¬ 
wide  savings  between  the  fiscal  years  2012  and  2016.  Army  leadership  was  provided  an 
annual  update  of  army  wide  services  acquisitions  requirements  and  cost  savings  through 
consolidation  of  the  49  Army  agencies  services  reports  by  the  SSM,  which  reported 

$2.29B  in  savings  as  of  IQTR  FY16,  Army  Senior  Services  Manager  Cost  Savings 

2 

Update,  December  21,  2015  (W.  Mercer,  personal  communication,  2016,  March  15) 

The  Cost  reduction  drivers  included  change  in  contract  types,  requirements  re¬ 
validation,  consolidations,  and  reduction  and  increased  competition  through  Strategic 
sourcing. 

An  Army  Services  Acquisition  Update  was  provided  to  the  Assistant  Secretary  of 
the  Army  for  Acquisition,  Logistics  and  Technology  (ASA(ALT))  on  12  May  2015  to 
assess  the  DASA(P)  Senior  Service  Manager  progress. 

The  ASA(ALT)  was  interested  in  how  the  goal  to  save  $2.5  billion  in  services 
acquisition  was  being  accomplished.  The  data  presented  by  the  SSM  reflected  that  52% 
of  the  cost  savings  were  achieved  by  changing  or  reducing  requirements  due  to  the 
increased  use  of  structured  requirements  and  conducting  requirements  validation  and 
reviews.  Additionally,  the  SSM  provided  the  quantities  of  service  acquisitions  that  have 
submitted  their  SAS  for  approval  in  Table  2  (ASA(ALT)  Services  Acquisition  Update 
briefing.  May  12,  2015;  W.  Mercer,  personal  communication,  August  18,  2015). 


^  A  PowerPoint  briefing  from  the  SSM  was  provided  via  email  by  a  portfolio  coordinator  at  the  Office 
oftheDASA(P)  SSM. 

3  A  PowerPoint  briefing  to  the  AS  A(ALT)  was  provided  via  email  by  a  portfolio  coordinator  at  the 
Office  of  the  DASA(P)  SSM. 
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Table  6.  Service  Acquisition  Strategy  Submissions.  Source:  W.  Mercer, 
personal  communication  (August  18,  2015) 


SAS  Submitted  to  SSM  (>$250M<$1B) 

Quantity 

Fiscal  Year 

10 

2013 

7 

2014 

3 

2015 

SAS  Submitted  to  DPAP  (>$1B) 

Quantity 

Fiscal  Year 

2 

2013 

5 

2014 

2 

2015 

Although  these  numbers  do  not  appear  to  be  significant  in  comparison  to  the 
$56.4  billion  in  2013  and  $48.8  billion  in  2014  spent  annually  by  the  Army  on  service 
acquisitions,  the  impact  to  the  specific  requirement  is  noteworthy  (ASA(ALT)  Services 
Acquisition  Update  briefing.  May  12,  2015;  W.  Mercer,  personal  communication,  August 
18,  2015).  This  section  will  provide  a  further  examination  into  the  mandatory 
documentation,  review  timeframes  to  complete  the  ASSP,  and  obtain  approval  of  their 
SAS. 

As  part  of  requirements  development,  the  Multi-Functional  Integrated  Product 
Team  (MFIPT)  is  required  to  participate  in  a  SAW.  DOD  requires  for  service  acquisitions 
meeting  or  exceeding  $1  billion,  however,  the  Army  has  a  more  stringent  requirement  for 
participation  in  a  SAW;  for  any  service  acquisition  $250  million  and  above,  and  strongly 
recommends  for  those  over  $10  million  (Office  of  the  Assistant  Secretary  of  the  Army, 
Acquisition,  Logistics  and  Technology,  2013).  During  the  SAW,  recommendation  will  be 
given  to  utilize  ARRT  to  help  develop  performance-based  requirements  for  these  service 
acquisitions.  Use  of  ARRT  is  not  mandatory,  but  the  DFARS  PGI  237.102-77  states  “this 
tool  should  be  used  ...”  While  the  SAW  can  be  helpful  in  training  the  MFIPT  in  using 
ARRT,  there  is  a  definite  learning  curve  to  understand  the  processes  of  the  system. 

In  a  typical  requiring/contracting  activity,  a  Solicitation  Review  Board  (SRB) 
must  be  conducted  prior  to  releasing  a  final  solicitation.  For  service  acquisitions  that  fall 
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under  the  description  provided  throughout  the  OSD  and  Army  policies  discussed,  many 
steps  are  to  be  completed  following  the  SRB  but  prior  to  releasing  the  final  solicitation. 

It  is  a  best  practice  that  the  requiring  activity  is  in  communication  with  their 
portfolio  coordinators  and  portfolio  managers  throughout  requirements  development  and 
all  the  way  through  submission  of  their  SAS  for  review/approval.  It  is  also  important  that 
the  IPT  publicize  their  upcoming  requirement  with  the  SSM  well  in  advance  to  ensure 
personnel  are  available. 

Prior  to  submitting  the  SAS  for  the  ASSP,  the  IPT  is  required  to  pre -brief  the 
Army  OSBP  (AFARS  5137.590-5).  There  may  also  be  pre -briefs  to  other  OSBP  offices 
prior  to  that  level  (such  as  the  Army  Materiel  Command)  as  requested,  but  not  per  policy. 
As  outlined  in  the  Secretary  of  the  Army  Office  of  Small  Business  Programs 
memorandum,  this  brief  is  required  to  address  the  small  business  opportunities 
specifically,  a  description  of  market  research  completed,  how  opportunities  and 
competition  are  increased,  how  small  business  participation  will  be  evaluated,  etc. 
(Secretary  of  the  Army  OSBP  2014).  The  OSBP  office  will  scrutinize  the  requirement 
and  validate  that  all  opportunities  were  pursued  where  possible.  The  required  ASSP 
documentation  includes  the  following  (ASSP  Meeting  list)  in  Table  7. 


Table  7.  DASA(P)  Service  Acquisition 


ASSP  Supporting  Documents  Required: 

Business  Case  Analysis 

Cost  Benefit  Analysis 

Completion  of  SAW  or  Waiver 

Consolidation  Determination  and  Findings  (D&F)  (if  applicable) 

Market  Research  Report 

Source  Selection  Authority  Appointment  (if  over  $500M,  by 
DASA(P) 

Performance  Work  Statement  (PWS) 

Quality  Assurance  Surveillance  Plan  (QASP) 

Small  Business  Coordination  Record  DD2579 

Independent  Government  Cost  Estimate 

Request  for  Services  Contract  Approval  Form 

Service  Acquisition  Strategy 

Source  Selection  Plan 

Pre-Brief  to  Army  Office  Small  Business  Programs 

ASSP  Briefing  Package 
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Although  the  Army  has  yet  to  provide  supplemental  guidance  in  response  to  the 
release  of  the  DODI  5000.74,  approval  for  service  acquisitions  exceeding  $1  billion  can 
be  delegated  to  an  Army  SAE  or  designee  (Department  of  Defense,  2016).  This  should 
reduce  review  and  approval  timelines  as  SAS  approval  stays  within  the  military 
department. 

According  to  a  portfolio  coordinator  from  the  SSM  office,  in  the  past  when 
submitting  a  package  for  DPAP  approval  they  estimated  60  days,  whereas  with  the 
approval  delegation,  they  anticipate  providing  approval  between  30  and  45  days.  (W. 
Mercer,  personal  communication,  March  16,  2016).^  To  ensure  best  success,  coordination 
with  the  Portfolio  Manager  and  Portfolio  Coordinator  throughout  requirements 
development  and  preparing  for  the  ASSP  is  recommended.  The  ASSP  supporting 
documentation  referenced  in  Table  3  would  already  have  been  completed  by  the  MFIPT 
prior  to  this  stage,  requiring  the  team  only  to  provide  an  ASSP  Briefing  as  an  additional 
requirement.  A  sample  ASSP  Briefing  provided  by  a  Portfolio  Coordinator  outlines  the 
briefing  requirements  are  included  in  Table  8. 


Table  8.  ASSP  Briefing  Contents  (Sample) 


Requirement 

Risk  Management  Approach 

Summary  of  Acquisition 

Small  Business  Provisions 

Acquisition  Benefits 

Business  Arrangements 

Previous  Acquisition  Challenges 

Source  Selection  Approach 

Organizational  Structure 

Source  Selection  Evaluation 

Market  Research 

Metrics 

Competition 

Acquisition  Schedule 

Consolidation 

Recommendation 

As  detailed  in  the  Chapter  IIB,  OSD  Policy  section,  DOD  requires  Peer  Reviews, 
both  for  pre-award  (up  to  three  phases)  and  post-award  (annually).  For  a  typical  Army 
acquisition  exceeding  $1  billion,  the  team  can  expect  approximately  66  additional  days 
(W.  Mercer,  personal  communication,  March  15,  2016)^  to  obtain  approval  from  the 

^  This  information  was  communicated  via  email  with  a  portfolio  coordinator  at  the  Office  of  the 
DASA(P)  SSM. 

^  This  information  was  communicated  via  email  with  a  portfolio  coordinator  at  the  Office  of  the 
DASA(P)  SSM. 
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Army  SAE  when  accounting  for  Army  OSBP  Pre-Brief,  anticipated  revisions,  the  routing 
of  the  SAS  and  the  completion  of  the  ASSP  Briefing.  The  process  begins  once  the  agency 
SRB  has  concluded  and  is  submitted  to  the  Army  SSM.  This  estimated  timeframe 
includes  the  Army  OSBP  pre -brief,  although  that  may  be  able  to  be  completed  in 
conjunction  with  the  SRB  (unrelated).  In  addition,  completion  of  the  Phase  1  OSD  Peer 
Review  is  anticipated  to  add  another  30  days  to  the  schedule  (W.  Mercer,  personal 
communication,  March  15,  2016). 

Once  the  ASSP  has  concluded  upon  receipt  of  SAS  approval,  the  IPT  is  permitted 
to  post  the  final  request  for  proposal  (AFARS  5137.590-5).  After  receiving  proposals  and 
evaluations  are  conducted,  a  Phase  2  OSD  Peer  Review  is  required  if  the  IPT  requests 
final  proposal  revisions.  This  could  lengthen  the  procurement  administrative  lead  time  by 
another  30  days. 

Finally,  after  a  local  agency  contract  review  board  provides  approval  of  the 
contract  award,  the  Phase  3  OSD  Peer  Review  is  required,  further  lengthening  the  process 
by  30  days  for  review  time.  (W.  Mercer,  personal  communication,  March  15,  2016). 

Table  9  reflects  the  total  estimated  additional  procurement  acquisition  lead  time  to 
conduct  the  OSD  and  Army  service  acquisition  review  requirements.  This  list  is  not 
inclusive  of  other  pre-briefs  that  could  include  subordinate  commands  (such  as  Army 
Materiel  Command)  or  the  SSM.  The  IPT  should  make  greatest  attempt  to  participate  in 
the  pre -briefs  in  conjunction  of  other  activities  without  lengthening  time  to  award. 


Table  9.  Timeline  of  Requirements  Leading  up  to  ASSP 


Activity 

Estimated  Duration 

Army  OSBP  Pre -brief 

21  days 

ASSP  Process 

45  days 

OSD  Phase  I  Peer  Review 

30  days 

OSD  Phase  2  Peer  Review 

30  days 

OSD  Phase  3  Peer  Review 

30  days 

Total  Estimated  Duration 

156  days 

One  might  question  why  briefings  would  require  such  an  extensive  period,  but  the 
IPT  must  account  for  coordination  days,  pre -reviews  (such  as  providing  briefing/ 
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supporting  documents  in  advance),  the  actual  briefing,  and  any  necessary  adjudications 
prior  to  final  approval.  In  all  of  these  review  instances,  the  Army  or  OSD  provides  a 
completion  memo,  and  the  IPT  is  unable  to  move  forward  until  receiving  that  approval. 
DASA(P)  Service  Acquisition. 

The  contracting  officer  is  required  to  document  the  disposition  of  all  Peer  Review 
recommendations  and  if  they  choose  not  to  accept  the  recommendations,  it  must  be 
addressed  why  they  were  not  incorporated.  The  signed  memorandum  is  to  be  provided 
prior  to  the  next  Peer  Review  (DFARS  PGI  201 . 170-4). 

Since  the  release  of  the  DODI  5000.74,  the  Army  is  hesitant  to  provide  specific 
guidance  on  approval  authorities,  as  the  Army  has  not  yet  announced  the  selection  of  the 
SAE.  This  affects  part  of  the  ASSP  process,  as  generally  when  a  pre-award  Peer  Review 
was  required,  DPAP  would  provide  the  final  approval  of  the  SAS.  Now  that  the  SAS 
approval  has  been  delegated,  there  is  some  uncertainty  in  the  process  (W.  Mercer, 
personal  communication,  March  17,  2016).^  The  following  flowchart  reflects  the  process 
and  requirements  of  the  ASSP. 

As  noted  in  the  ASSP  Requirements  (Figure  5),  the  SSM  has  the  ability  to  waive 
the  ASSP  briefing  requirement  if  the  package  submittal  contents  have  thoroughly 
explained  and  addressed  the  requirements  and  no  further  questions  necessitate  a  briefing 
(AFARS  5137.590-5(a)(2)).  If  the  MFIPT  thoroughly  followed  the  instructions  provided 
“prior  to  the  ASSP,”  the  likelihood  may  be  higher.  For  example;  if  a  team  utilizes  ARRT 
to  create  their  performance-based  requirements,  they  should  be  better  suited  for  the 
acquisition  and  could  therefore  result  in  reduced  revisions.  Additionally,  constant 
communication  and  sharing  of  requirements  documents  with  the  Portfolio  Coordinators 
and  Portfolio  Managers  will  allow  them  the  opportunity  to  provide  suggestions  prior  to 
the  ASSP  briefing. 


6  This  information  was  communicated  to  use  via  email  with  a  portfolio  coordinator  at  the  Office  of  the 
DASA(P)  SSM. 
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Figure  5.  Army  Services  Strategy  Panel  (ASSP)  Requirements  for  Service  Acquisitions  >  $250M 
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There  are  two  briefings  required  per  the  ASSP  Requirements  Figure  5  that  are  not 
part  of  any  current  Army  policies:  the  SSM  pre -brief  and  the  AMC  OSBP  pre -brief  (if 
aligned  under  AMC).  The  SSM  pre -brief  is  something  that  the  Portfolio  Coordinators 
request  the  MFIPT  to  provide  an  overview  of  the  acquisition  and  the  anticipated  strategy. 
This  is  to  be  conducted  one  to  two  months  prior  to  beginning  the  ASSP  process  (W. 

■7 

Mercer,  personal  communication,  March  15,  2016).  The  AMC  OSBP  pre -brief  is  now 
requested  so  they  have  insight  into  the  acquisition  strategy  prior  to  briefing  the  Army 
OSBP  where  small  business  approves  of  the  acquisition  strategy  prior  to  beginning  the 
ASSP  (W.  Mercer,  personal  communication,  March  16,  2016).  Recommend  these 
additional  requirements  will  be  part  of  the  updated  Army  policy  (AR  70-13)  in  response 
to  the  release  of  the  DODI  5000.74 

BBP  Analysis 

Direction  provided  in  BBP  acquisition  improvement  initiatives  drives  policy 
change  across  the  DOD.  Each  branch  adopts  and  creates  practices  to  help  achieve  these 
goals  in  a  manner  that  works  best  within  the  organization.  The  tenants  outlined  in  BBP 
are  filtered  down  and  impressed  upon  acquisition  communities  Many  of  the  goals 
identified  in  BBP  are  fairly  straightforward  in  nature  when  assessed  individually.  In  a 
vacuum,  it  would  be  easy  to  create  policies  to  achieve  certain  initiatives  without  factoring 
for  the  side  effects  of  those  changes  and  how  they  might  impact  other  areas  of  the 
acquisition  process.  Analyzing  BBP  direction  shows  areas  where  intentions  may 
contradict  each  other. 

BBP  strives  for  achieving  affordable  programs  with  respect  to  cost,  schedule  and 
manpower.  The  DOD  is  constantly  resource  challenged  so  it’s  understandable  why  this 
would  be  a  goal.  The  Army  services  acquisition  review  process  is  designed  to  help 
facilitate  procurements  that  will  provide  the  best  value  to  the  government  and  the 
taxpayer.  It’s  important  to  realize  though  that  these  review  processes  require  greater 
government  resources  in  order  to  be  effective.  The  man-hours  and  associated  cost  to 


^  This  information  was  communicated  to  use  via  email  with  a  portfolio  coordinator  at  the  Office  of  the 
DASA(P)  SSM. 
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conduct  and  complete  these  reviews  should  be  tracked  and  evaluated  to  ensure  value  add 
is  realized. 

BBP  also  aims  to  eliminate  unproductive  processes  and  bureaucracy  to  eliminate 
low  value-added  processes.  To  do  so,  individual  commands  needs  to  exercise  more 
autonomy  and  manage  programs  organically  rather  than  mandating  higher  level  reviews. 
To  determine  whether  or  not  review  processes  are  providing  critical  inputs  to  the 
procurement,  the  Army  must  constantly  assess  whether  the  input  requirements  to 
participate  in  the  peer  reviews  warrant  the  end  result.  Without  a  critical  look  at  the 
process  and  realized  gains,  a  determination  cannot  be  made  with  respect  to  the  value  of 
the  process. 

Currently,  DOD  requires  DPAP  Peer  Reviews  for  service  acquisitions  exceeding 
$1B;  however,  with  the  release  of  DODI  5000.74,  the  USD(AT&L)  has  the  ability  to 
delegate  the  approval  of  the  Acquisition  Strategy  to  the  Army  SAE.  Considering  that  the 
$1B  review  threshold  now  has  the  ability  to  be  waived,  the  DOD  should  consider 
increasing  the  threshold  for  OSD  Peer  Reviews.  Requirements  for  seeking  Army  SAE 
review  delegation  could  also  be  outlined  to  reduce  more  OSD  level  reviews. 

The  Army  further  mandates  that  service  contract  requirements  in  excess  of 
$250M  must  participate  in  the  ASSP  process.  The  Army  could  also  consider  increasing 
this  monetary  threshold  to  reduce  the  resource  challenges  associated  with  higher  levels 
reviews. 

Part  of  the  primary  focus  of  the  ASSP  review  process  is  to  help  promote  effective 
competition  for  Army  requirements.  This  coincides  with  BBP  initiative  to  not  only 
promote  but  to  maintain  competitive  environments  and  create  opportunities  for  small 
businesses.  To  achieve  this  goal,  extensive  resources  must  be  applied  during  the 
acquisition  planning  process  to  ensure  that  the  Army’s  requirements  can  be  met  without 
creating  overly  restrictive  solicitation  requirements.  The  Army  should  track  and  assess 
whether  or  not  the  goals  are  being  obtained  and  weigh  the  perceived  benefits  against  the 
resource  requirements. 
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BBP’s  focus  on  improving  the  tradecraft  in  the  acquisition  of  services  also 
influences  Army  contracting  policies.  The  ASSP  review  process  helps  facilitate  improved 
acquisition  approaches  and  implementation  by  following  the  required  OSD  level 
oversight  and  influence.  This  also  comes  with  an  associated  cost  in  terms  of  man-hours 
and  schedule  requirements.  The  Army  should  assess  the  resulting  benefits  and 
improvements  after  completion  of  requirements  reviews. 

Improvement  of  the  acquisition  workforce  will  continue  to  be  a  pillar  of  BBP.  It 
will  take  time  and  considerable  efforts  to  make  the  wide  spanning  culture  change  to 
develop  and  retain  a  talented  acquisition  community.  As  these  changes  occur  over  time, 
the  Army  should  track  whether  OSD  level  reviews  are  still  necessary  and  provide 
sufficient  value  as  compared  to  schedule  and  cost  requirements.  This  could  provide 
insight  to  OSD  to  consider  if  the  reviews  could  be  pushed  back  to  the  military 
departments.  As  the  acquisition  community  grows  and  evolves,  more  autonomy  should 
be  applied  resulting  in  less  peer  reviews  while  maintaining  improved  acquisition 
documents. 
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IV.  RESEARCH  METHODOLOGY 


The  section  will  leverage  existing  research  in  order  to  create  a  tool  to  analyze  the 
Army  service  acquisition  review  requirements  and  test  the  perceived  effectiveness  of  the 
intended  policy  improvements.  In  order  to  gain  insight  from  participants  of  the  ASSP 
review  process,  a  survey  will  be  developed  for  potential  use  by  DOD  leadership.  As  part 
of  the  development  process,  the  tool  will  be  field  tested  by  a  group  that  represents  the 
target  population  for  review,  and  comments  for  potential  improvements.  In  order  to  create 
an  effective  instrument,  it  is  critical  that  proper  survey  development  techniques  are 
implemented.  Various  scholastic  resources  were  reviewed  with  slightly  varying 
methodology  to  provide  a  framework  to  navigate  this  process.  Components  of  the 
different  methods  will  be  utilized  in  order  to  develop  an  effective  survey  instrument  that 
will  best  pull  information  needed  to  meet  the  stated  objective.  This  section  will  look 
closely  at  elements  of  various  surveying  techniques  and  apply  them  in  this  context. 

A.  SURVEY  DEVELOPMENT  PROCESS 

Fundamentally,  a  determination  needs  to  be  made  whether  to  utilize  a 
quantitative  or  qualitative  methodology,  which  will  set  the  foundation  for  the  survey  as 
a  whole.  The  two  different  approaches  have  varying  ideologies,  which  will  impact  each 
step  in  the  survey  development  process. 

Quantitative  methodology  refers  to  an  approach  involving  the  use  of 
numeric -based  information  that  can  be  measured,  compared,  and  analyzed 
statistically.  This  methodology  is  primarily  used  as  a  way  to  quantify 
achievement  of  outcomes  as  a  result  of  participation  in  a  program.  The 
advantage  of  quantitative  methodology  is  that  it  measures  the  reactions  of 
a  great  many  people  to  a  limited  set  of  questions,  thus  facilitating 
comparison  and  aggregation  of  data.  Therefore,  findings  can  be 
generalized. 

Qualitative  methodology  refers  to  an  approach  that  examines,  describes,  or 
interprets  a  program.  Qualitative  methods  make  it  possible  to  study  a 
program  or  issue  in  more  depth  and  detail  than  quantitative  methods.  This 
methodology  is  designed  to  find  out  what  people  do,  plan  to  do,  think,  and 
feel. 
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Qualitative  methods  provide  vast  amounts  of  detailed  information  about 
small  numbers  of  people  and  cases,  while  quantitative  methods  can  deal 
with  large  numbers  of  people  and  limited  sets  of  questions.  (Barkman, 

2002,  p.  9). 

For  the  purposes  of  answering  the  primary  research  question,  a  qualitative  survey 
instrument  will  be  developed. 

The  next  step  in  the  process  is  to  identify  what  will  be  accomplished  by  issuing 
the  survey.  Before  designing,  it  is  essential  to  identify  the  objective,  or  the  reason  the 
survey  is  being  conducted  (Loyola,  n.d.).  Understanding  the  objectives  sets  the 
framework  and  narrows  in  the  focus  and  direction  of  the  survey. 

It  is  also  critical  to  determine  what  is  being  measured.  This  should  be  decided 
based  on  the  overall  research  objectives  and  will  also  frame  in  latter  parts  of  the  survey. 

Surveys  typically  measure  any  of  the  following:  attitudes  or  perceptions; 
knowledge;  skills;  goals,  intentions,  aspirations;  behavior  and  practices; 
knowledge,  skills  and  behaviors.  It’s  possible  to  measure  more  than  one  of 
these  attributes,  but  questions  will  differ  in  wording  and  response  options 
based  on  what  is  being  targeted.  (Rutgers,  2002,  p.  1-2). 

For  the  purpose  of  accomplishing  the  thesis  objectives,  the  survey  instrument  will 
be  developed  to  assess  attitudes  or  perceptions,  which  carry  similar  design  characteristics. 

Another  important  decision  point  in  the  survey  development  is  to  choose  an 
appropriate  data  collection  procedure.  Particularly,  emphasis  should  be  placed  on 
deciding  whether  or  not  to  use  confidential  or  anonymous  methodology.  Before  making 
this  determination,  it  is  important  to  understand  the  fundamental  differences  between  the 
two  (Rutgers,  2002). 

Utilizing  confidential  collection  techniques  helps  protect  identities  during 
surveying.  Characteristics  of  confidential  collection  include:  names  and 
identifiers  used  to  follow  non-respondents  or  match  data  from  pretests; 
information  is  not  used  for  other  purposes;  and  identifying  information  is 
destroyed  after  survey  is  complete. 

Anonymous  collection  methods  also  help  protect  the  identities  of  the 
respondents.  Characteristics  of  anonymous  collection  methods  include: 
name  is  not  asked  of  respondents  (researcher  is  unable  to  follow  non¬ 
respondents  or  match  data  from  pretests);  and  can  collect  basic  descriptive 
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data  (can  be  useful  for  comparing  respondents  with  the  population). 
(Rutgers,  2002,  p.  2-3). 

For  the  purposes  of  the  survey  being  developed,  an  anonymous  survey  would  be 
most  appropriate.  Considering  that  the  survey  is  measuring  pereeptions  of  individuals 
work  environments  as  well  as  oversight  officials,  anonymity  is  key  to  getting  honest 
answers.  Further,  demographic  information,  such  as  job  category  eould  be  useful 
information  to  collect. 

Knowing  the  survey  foeus  intends  to  assess  attitudes  or  perceptions,  the  survey 
should  be  constructed  to  assess  altitudinal  outcomes. 

When  trying  to  measure  the  type  of  outcome,  there  are  three  types  of 
attitudinal  statements.  Cognitive  items  express  beliefs.  Affective  Items 
express  feelings.  Conative  items  express  behavior  intention  or  preference. 
(Barkman,  2002,  p.  22). 

Considering  the  survey  itself  will  assess  DOD  employees’  perspective  on  reeent 
policy  changes,  the  foeus  should  be  on  cognitive  items  as  the  instrument  would  be  erafted 
to  understand  the  individual’s  beliefs  or  opinions  of  the  subject. 

Response  options  must  be  crafted  in  a  manner  whieh  will  support  a  cognitive 
attitudinal  assessment.  Failure  to  properly  strueture  the  questionnaire  will  result  in 
inconsistent  or  flawed  data.  The  model  followed  for  this  survey  will  be  eonstrueted  as 
demonstrated  below: 

Respondents  can  rate  statements  about  the  topic  on  a  scale  such  as: 
strongly  disagree,  disagree,  undecided,  agree,  and  strongly  agree.  The 
mean  (average)  rating  of  those  statements  indicates  the  respondent’s 
attitude  toward  the  topic.  (Barkman,  2002,  p.  22). 

An  example  of  this  type  of  statement  and  response  set  can  be  found  in  Figure  6. 
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Examples  of  Attitude  Questions  and  Response  Choices 


Please  circle  the  letter  that  corresponded  to  your  feelings  about  each  of  the  statements  below. 


STRONGLY 

DISAGREE 

DISAGREE 

UNDECIDED 

AGREE 

STRONGLY 

AGREE 

1  would  serve  genetically  modified 

foods  to  my  family. 

a 

b 

C 

d 

e 

Eating  genetically  modified  foods  is 

harmful  to  a  person’s  health. 

a 

b 

C 

d 

e 

Genetically  modified  foods  are  inferior 
to  foods  from  plants  developed  using 
conventional  methods. 

a 

b 

C 

d 

e 

1  would  eat  foods  containing  products 
derived  from  using  biotechnology 
research. 

a 

b 

C 

d 

e 

NOTE:  Scoring  for  this  test  would  be  as  follows:  a  =  1 ,  b  =2,  c  =3.  d  =  4,  and  e  =  5.  The  lower 
the  mean  (average)  of  the  respondent’s  score,  the  more  negative  their  attitude  toward 
genetically  modified  foods. 

But  remember,  you  need  to  use  reverse  coding  for  tiegatively  worded  statements. 
Scoring  for  negatively  worded  questions  would  be:  a  =  5,  b  =4.  c  =3,  d  =  2.  and  e  =  1 . 
So,  if  a  lespotident  answered  “e”  on  question  2,  you  would  score  it  as  a  “1”  not  a  “5”. 


Figure  6.  Attitude  Questions  and  Response  Choices.  Source:  Barkman  (2002) 


This  survey  type  aligns  well  in  with  the  project  objective.  By  using  this  response 
type  and  scoring  system  an  aggregate  of  response  averages  can  be  quickly  recorded  for 
individual  questions. 

B.  ESTABLISHING  SURVEY  OBJECTIVES 

The  survey  itself  will  be  designed  to  help  determine  a  set  of  primary  objectives 
centered  around  assessing  Army  service  acquisition  personnel’s  perceptions  about  the 
service  contracting  review  board  requirements.  The  objectives  have  been  derived  after  the 
extensive  review  of  OSD,  Army,  GAO  and  BBP  policies  that  helped  shape  the  review 
process  intent  and  focus.  The  objective  of  the  survey  is  to  create  an  instrument  that  will 
help  to  shed  insight  on  the  following  five  questions: 
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1.  Is  the  ASSP  review  process  leading  to  desired  outcomes? 

2.  Does  value  add  of  ASSP  reviews  exceed  resource  outputs  required  to 
complete  process? 

3.  Are  proper  resources  available  prior  to  ASSP  review  in  order  to 
maximize  value? 

4.  Are  commands  capturing  metrics  to  document  ASSP  review  process 
impacts? 

5.  Are  commands  taking  appropriate  measures  to  capture  lessons  learned? 

The  survey  instrument  provides  a  vehicle  to  gain  insight  from  the  perspectives  of 
the  reviews’  participants.  Answering  these  fundamental  questions  will  help  to  determine 
the  effectiveness  of  the  Army’s  services  acquisition  review  processes,  and  will  help 
provide  feedback  to  DOD  leadership  on  potential  ways  to  maximize  value  for  services 
acquisitions.  Given  the  time  required  to  successfully  complete  the  Army’s  services 
requirements,  it’s  important  to  take  a  critical  look  at  the  process. 

1.  Is  the  ASSP  Review  Process  Leading  to  Desired  Outcomes? 

Of  the  four  challenges  identified  by  GAO  as  high  risk,  the  most  significant  issue 
is  the  lack  of  an  action  plan  to  guide  DOD’s  efforts  when  acquiring  services.  DOD  was 
unable  to  communicate  their  plan  to  improve  the  acquisition  of  services,  identify 
necessary  changes  and  processes  to  prioritize  risks  or  even  understand  service  spending 
trends  (Cooper,  2005).  Additionally,  since  the  early  2000’s,  a  repeat  finding  by  GAO  was 
that  service  procurements  were  not  being  done  efficiently.  As  stated  previously, 
“agencies  were  not  clearly  defining  requirements,  fully  considering  alternative  solutions, 
performing  vigorous  price  analyses,  and  adequately  overseeing  contractor  performance” 
(Cooper,  2001,  p.  1). 

To  help  improve  the  contracting  process,  the  Army  established  the  ASSP  review 
requirements.  The  Army  has  a  series  of  pre-established  goals  that  they  intend  to  achieve 
by  mandating  that  certain  procurements  complete  this  review.  As  a  primary  focus  of  the 
survey,  a  series  of  statements  and  answer  responses  will  be  crafted  to  help  gain 
understand  whether  ASSP  participants  believe  these  goals  are  being  obtained. 
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2.  Does  the  Value  Add  of  ASSP  Reviews  Exceed  Resource  Outputs 
Required  to  Complete  the  Process? 

In  an  effort  to  ensure  the  resource  outputs  from  DOD  are  value  added,  DOD 
needs  to  adopt  a  strategy  and  develop  guidance  that  could  provide  better  management  and 
oversight  and  help  leverage  its  buying  power.  Cooper  recommended  “key  elements  of 
this  guidance  should  address: 

•  Improving  knowledge  of  services  spending  of  services  spending  by 
collecting  and  analyzing  data  about  services  procurements  across  DOD 
and  within  military  departments  and  defense  agencies, 

•  Promoting  collaboration  across  DOD  and  within  military  departments  and 
defense  agencies  promoting  collaboration  across  DOD  and  within  military 
departments  and  defense  agencies  by  establishing  cross-functional  teams 
to  carry  out  coordinated  purchasing  of  services,  and 

•  Establishing  strategic  savings  and  performance  goals,  measuring  results, 
and  ensuring  accountability  by  assigning  high-level  responsibility  for 
monitoring  those  results”  (Cooper,  2003,  p.  14) 

While  addressing  these  key  elements,  the  ASSP  requires  a  significant 
commitment  from  the  personnel  that  are  mandated  to  complete  the  process  as  well  as  the 
oversight  officials  involved  in  the  reviews.  Consequently,  because  DOD  is  constantly 
faced  with  resource  restrictions,  it  is  imperative  to  take  a  look  at  the  effectiveness  of  the 
reviews. 

Statements  and  responses  in  the  survey  will  be  tailored  to  help  shed  light  from 
recent  participants.  If  input  requirements  are  found  to  exceed  the  value  add  associated 
with  completing  the  review,  consideration  should  be  given  to  addressing  the  issue. 

3.  Are  Proper  Resources  Available  Prior  to  ASSP  Review  in  Order  to 
Maximize  Value? 

Specific  to  the  Army,  GAO  reported  that  the  lack  of  trained  personnel  lead  to 
inadequate  contract  oversight.  In  some  cases.  Army  surveillance  was  a  collateral  duty  for 
personnel.  “To  improve  surveillance  and  further  mitigate  risk,  GAO  recommended  that 
the  Secretary  of  Defense  ensure  proper  surveillance  training  of  personnel,  develop 
practices  for  accountability,  and  ensure  the  service  contract  review  process  and  associated 
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data  collection  requirements  provide  information  that  will  provide  more  management 
visibility  over  contract  surveillance”  (Cooper,  2005,  p.l6).  The  Army,  in  an  effort  to  ease 
coordination,  planned  to  adopt  a  strategy  on  how  to  centralize  acquiring  services,  rather 
than  being  compartmentalized  by  consolidating  contracting  activities  to  reduce  costs  and 
centralize  services  acquisitions. 

Given  that  the  ASSP  is  a  relatively  new  requirement,  it  is  reasonable  to  expect 
many  participants  may  be  inexperienced  with  the  process.  Considering  this  constraint,  it’s 
important  to  understand  whether  or  not  oversight  officials  or  experienced  personnel  are 
available  to  help  guide  and  direct  the  acquisition  team  during  the  early  phases  of  the 
procurement.  Without  sufficient  resources,  valuable  time  and  effort  could  be  wasted  to 
potential  rework.  The  survey  will  help  to  understand  individual  participant’s  perspective 
about  their  support  structure. 

4.  Are  Command  Capturing  Metrics  to  Document  ASSP  Review  Process 
Impacts? 

Coinciding  with  the  first  survey  question,  it’s  important  to  document  whether  or 
not  ASSP  objectives  are  being  met.  Additionally,  it’s  important  for  commands  to  track 
metrics  to  understand  the  impacts  of  the  review  process  in  terms  of  cost,  schedule,  and 
programmatic  risk. 

DiNapoli  reports  that  DOD  does  not  utilize  available  data  or  metrics  to  determine 
trends  or  define  goals  which  would  put  them  in  a  better  position  to  track  progress  or 
determine  impacts  to  services  acquisition.  To  do  this,  DiNapoli  recommended  “that  the 
Principal  Deputy  USD  (AT&L),  in  consultation  with  the  military  departments’  SSMs, 
take  the  following  three  actions: 

•  Identify  baseline  data  on  the  status  of  service  acquisition 

•  Develop  specific  goals  associated  with  their  actions  to  improve  service 
acquisition,  and 

•  Establish  metrics  to  assess  progress  in  meeting  these  goals”  (DiNapoli, 
2013,  p.  21) 
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Statements  and  responses  will  be  created  in  the  survey  to  show  whether  or  not 
individual  commands  are  tracking  the  outcomes  and  effects  of  the  ASSP  process. 

5.  Are  Commands  Taking  Appropriate  Measures  to  Capture  Lessons 
Learned? 

As  Francis  stated,  because  DOD’s  approach  is  reactive  when  acquiring  services, 
they  remain  “at  high  risk  of  vulnerabilities  to  fraud,  waste,  abuse  and  mismanagement” 
(Francis,  2006,  p.  1).  The  ASSP  review  process  requires  the  acquisition  team’s  personnel 
to  complete  a  stringent  review  process  with  oversight  from  senior  acquisition 
professionals.  In  his  GAO  report,  Francis  recommended  a  more  proactive  approach  to 
managing  service  acquisitions  by  “capturing  lessons  learned  and 

•  establish  a  normative  position  of  how  and  where  service  acquisition 
dollars  are  currently  and  will  be  spent  (including  volume,  type,  and 
trends); 

•  determine  areas  of  specific  risk  that  are  inherent  in  acquiring  services  and 
that  should  be  managed  with  greater  attention  (including  those  areas 
considered  sensitive  or  undesirable  in  terms  of  quantity  or  performance); 

•  on  the  basis  of  the  above,  clearly  identify  and  communicate  what  service 
acquisition  management  improvements  are  necessary  and  the  goals  and 
timelines  for  completion; 

•  ensure  that  decisions  on  individual  transactions  are  consistent  with  DOD’s 
strategic  goals  and  objectives; 

•  ensure  that  requirements  for  individual  service  transactions  are  based  on 
input  from  key  stakeholders;  and 

•  provide  a  capability  to  determine  whether  service  acquisitions  are  meeting 
their  cost,  schedule,  and  performance  objectives”  (Francis,  2006,  p.  34) 

It  is  important  that  lessons  learned  from  senior  leadership  are  realized  and  applied 
toward  future  procurements.  The  survey  will  seek  to  gain  perspective  on  whether  or  not 
lessons  learned  are  being  distributed  across  acquisition  programs. 
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C.  CREATING  THE  SURVEY 

The  survey  was  created  using  design  approaches  identified  during  the  research 
phase.  The  instrument  was  constructed  to  help  answer  the  five  primary  survey  objectives 
by  creating  a  series  of  statements  and  response  options  that  could  be  aggregated  to 
provide  insight  into  the  actual  perceptions  of  those  who  have  been  through  the  ASSP 
process.  Statements  were  developed  in  manner  that  addressed  specific  elements  of  recent 
policy  changes  to  help  assess  the  end  results. 

Individual  statements,  along  with  the  rationale  behind  their  creation  are  provided 
in  Appendix  1.  Appendix  1  applies  DOD  and  Army  policies  against  the  each  statement, 
to  demonstrate  their  significance.  A  number  of  these  statement  and  response  options  were 
designed  to  help  provide  insight  where  policies  may  seem  to  contradict. 

Statements  outlined  the  Appendix  1  are  combined  into  the  draft  survey 
instrument.  Appendix  2.  Each  statement  has  a  corresponding  answer  set  that  is  designed 
as  a  Likert  scale  to  measure  varying  degrees  of  agreement  with  the  individual  questions. 
The  response  options  range  from  Strongly  Agree,  Agree,  Neutral,  Disagree,  and  Strongly 
Disagree.  An  option  is  also  provided  to  choose  not  applicable  if  the  respondent  were  to 
determine  that  a  question  is  not  relevant  to  their  experience.  This  statement  and  answer 
design  will  help  assess  each  individual’s  perceptions  on  the  five  primary  research 
questions. 

D.  PILOT  TESTING 

After  creating  the  survey,  it’s  critical  to  validate  or  pilot  test  the  instrument.  A 
pilot  test  provides  an  opportunity  for  an  unbiased  party  to  review  and  critique  the 
instrument.  It  can  also  greatly  increase  the  accuracy  and  consistency  of  the  survey. 

This  process  can  help  to  demonstrate  whether  or  not  the  questions  and 
response  choices  are  easy  to  understand  and  adequate.  Misleading  or 
unclear  questions  can  also  be  pointed  out  during  the  pilot.  The  process 
also  provides  an  opportunity  for  feedback  and  suggestions  for 
improvement  from  an  individual  who  falls  within  the  demographic  that 
would  otherwise  have  been  surveyed.  (Barkman,  2002,  p.  31). 
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The  pilot  testing  process  provides  the  researchers  valuable  critiques  from  an 
unbiased  party  that  represents  a  sample  of  the  targeted  population.  During  this  phase, 
decisions  should  be  made  whether  or  not  to  revise  or  improve  statement,  or  potentially 
remove  some  altogether.  This  feedback  should  help  to  make  the  survey  instrument 
stronger  and  ready  for  implementation. 

In  order  to  pilot  test  this  survey,  a  draft  version  of  the  survey  was  provided  to  a 
group  of  Army  acquisition  personnel  who  have  previously  experienced  the  ASSP  review 
process.  A  listing  of  acquisitions  that  had  gone  through  the  ASSP  review  process  was 
provided  by  a  Portfolio  Coordinator  at  the  Office  of  the  DASA(P)  SSM  (W.  Mercer, 

o 

personal  communication,  August  18,  2015).  This  information  was  then  utilized  to  pull 
the  points  of  contact  from  the  Federal  Business  Opportunities  website  (Federal  Business 
Opportunities)  to  send  requests  for  participation  in  this  pilot  test.  Appendix  3  provides  the 
initial  correspondence  sent  by  the  team  to  those  interested  in  participating  in  the  pilot  test. 
The  initial  email  was  sent  on  January  27,  2016  providing  a  brief  summary  of  the  project 
with  the  intent  to  submit  a  survey  to  be  field  tested  in  the  March  timeframe.  The  survey 
was  provided  on  March  7,  2016  to  all  possible  participants  to  include  their  program  and 
contracts  personnel  from  working  level  to  management.  This  sample  population  is 
representative  of  the  demographic  that  would  be  targeted  should  the  Army  issue  the 
actual  survey  at  a  later  date.  This  process  injects  valuable  firsthand  perspective  and 
suggestions  on  ways  to  improve  the  effectiveness  of  the  instrument. 

Instructions,  provided  as  Appendix  2,  were  provided  to  program  participants 
supporting  the  pilot  test.  The  pilot  will  allow  each  participant  approximately  two  weeks 
to  review  the  survey  and  provide  comments  or  suggestions  on  overall  survey 
improvement.  The  results  of  the  pilot  test  will  be  summarized  and  recorded  in 
Appendix  1.  Inputs  and  feedback  will  be  weighed  and  factored  into  the  final  survey 
design. 


^  This  information  was  communicated  to  use  via  email  with  a  portfolio  coordinator  at  the  Office  of  the 
DASA(P)  SSM. 
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E.  ASSESSING  PILOT  TESTING  FEEDBACK 


The  field  test  resulted  in  responses  from  4  individuals,  which  means  that  half  of 
those  that  the  survey  was  sent  to,  took  the  time  to  review  the  tool  and  provide  comments. 
Ideally,  the  team  would  have  preferred  a  higher  response  rate.  Time  constraints  and 
workloads  of  field  test  participants  made  requesting  an  additional  data  call  not  feasible. 
Despite  the  low  response  rate,  valuable  contributions  were  received  from  those  that  did 
participate.  The  feedback  received  during  the  pilot  testing  process  was  evaluated  by  the 
team  to  determine  how  to  best  utilize  the  information.  Individual  comments  and 
suggestions  provided  critical  insight  to  help  improve  the  overall  survey  design. 

The  respondents  all  provided  varying  feedback  on  the  survey.  Three  of  the  four 
elected  to  answer  all  of  the  survey  questions  to  demonstrate  how  they  would  respond  to 
the  question  as  written.  Further,  these  three  provided  additional  comments  as  they 
addressed  the  individual  questions  to  help  clarify  their  positions.  The  forth  respondent  did 
not  indicate  a  response  for  each  question  to  provide  their  answer  as  written.  This 
individual  did  however  provide  written  feedback  to  help  improve  certain  questions. 

The  results  of  the  field  test  feedback  are  summarized  in  Appendix  1  and  briefly 
discussed  in  the  next  chapter.  Comments  from  the  field  test  participants  were  reviewed 
and  assessed  to  effectively  utilize  the  critiques.  The  team  made  an  assessment  on  each  of 
the  comments  and  ultimately  made  a  determination  on  how  to  revise  the  survey 
instrument.  The  influence  of  the  field  test  helped  create  a  more  comprehensive  and 
effective  tool. 

F.  FINAL  SURVEY  INSTRUMENT 

The  revised  and  final  survey  instrument  is  provided  as  Appendix  2.  This  survey  is 
the  result  of  in  depth  policy  analysis  and  critical  design  efforts.  The  goal  of  the  survey 
design  was  to  improve  individual  statements  to  help  create  the  more  polarizing  responses. 
This  was  achieved  by  adjusting  verbiage  and  revising  multi-tiered  questions  to  ensure 
primary  focus  is  evident.  The  team  also  aimed  to  minimize  questions  that  are  truly  not 
applicable. 
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This  product  has  been  improved  through  the  process  of  field-testing  individuals 
that  have  submitted  requirements  packages  through  the  ASSP  process  and  is  shaped  to 
become  an  informative  tool  for  Army  leadership.  The  survey  can  be  utilized  to  provide 
critical  feedback  and  inputs  to  help  develop  an  action  plan  or  strategy  to  guide  efforts  in 
services  acquisition  and  help  craft  future  policy  decisions. 
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V.  FINDINGS  AND  RESULTS 


This  section  reviews  the  findings  and  results  of  the  survey  instrument  as  provided 
to  program  participants  supporting  the  pilot  test.  As  stated  earlier,  the  objective  of  the 
survey  is  to  create  an  instrument  that  will  help  to  shed  some  insight  on  the  following  five 
questions: 

1.  Are  the  Army  service  acquisition  requirements/reviews  leading  to  desired 
outcomes? 

2.  Does  the  value  added  through  completion  of  the  requirements/review 
exceed  the  resource  outputs  required  to  complete  the  process? 

3.  Is  proper  training/mentorship  available  prior  to  Army  service  acquisition 
requirements/reviews  to  maximize  the  value  of  the  ASSP? 

4.  Are  Commands  capturing  metrics  to  document  the  requirements/reviews 
impacts? 

5.  Are  Commands  taking  appropriate  measures  to  capture  lessons  learned? 

The  finding  and  results  for  each  objective  are  summarized  below.  Detailed  results 
of  the  pilot  test  are  recorded  in  Appendix  1. 

Are  the  Army  service  acquisition  requirements/reviews  leading  to  desired 

outcomes? 

The  statements  developed  to  address  this  question  focused  in  the  perceived 
benefit  of  increased  oversight  and  the  impact  on  the  following: 

•  Services  contract  spending 

•  Small  business  contracting  opportunities 

•  Utilization  of  incentive  based  contracts 

•  Requirements  definitions 

•  Effective  competition 

Additionally,  statements  were  developed  that  address  the  influence  of  the  process 
over  increased  critical  thinking  toward  the  acquisition  approach,  early  identification  of 
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potential  risk  for  proposed  strategies,  and  the  whether  or  not  the  Army  Service 
Acquisition  Requirements  reviews  help  to  prevent  a  protest  from  being  sustained. 

Overall,  the  results  varied.  While  some  statements  may  have  elicited  strong 
responses,  others  were  neutral  or  seen  as  inapplicable  to  their  organization.  In  instances 
where  the  respondents  were  unsure  of  how  to  respond  to  the  statement,  the  statement  was 
revised  to  be  clearer,  with  the  intent  to  elicit  a  stronger  response. 

Specific  to  the  statement  regarding  the  utilization  of  incentive -based  contracts, 
individual  respondents  did  not  believe  the  statement  to  be  applicable  to  their 
procurements.  Responses  indicate  that  this  objective  may  not  be  understood  or  it  is  not 
being  achieved.  In  this  instance,  the  statement  was  not  revised;  the  intent  was  to 
determine  whether  or  not  there  is  an  increased  use  of  incentive-based  contracts.  This 
statement  will  help  to  provide  feedback  to  the  Army  to  assess  whether  or  not  the  review 
process  is  aiding  to  achieve  this  goal.  For  those  statements  that  elicited  strong  responses, 
there  was  no  change  required. 

Does  the  value  added  through  completion  of  the  requirements/review 

exceed  the  resource  outputs  required  to  complete  the  process? 

The  statements  developed  to  address  this  question  focused  on  the  following: 

•  Improvements  in  acquisition  approach,  skillsets  of  acquisition  team, 
procurement  outcomes  and  acquisition  strategies 

•  Opportunities  for  feedback  for  the  acquisition  team 

•  Schedule  risks  on  programs 

•  Utilization  of  ARRT 

•  Consequent  delays 

•  Value  add  to  the  services  acquisition  process,  such  as  best  practices  and 
capturing  lessons  learned 

Similar  to  the  first  question,  the  results  of  the  statements  addressing  this  question 
varied.  While  the  majority  of  the  questions  elicited  strong  enough  responses  and  the 
statements  remained  unchanged,  there  were  two  statements  that  were  revised. 
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The  first  statement  revised  addressed  whether  the  time  spent  preparing  for  and 
participating  in  the  OSD  Pre-Award  Peer  Review  provided  valuable  input  to  the 
acquisition  resulting  in  improvements  to  the  Acquisition  Strategy.  The  statement  did  not 
elicit  strong  responses  as  written.  Because  it  is  important  to  determine  whether  the 
additional  oversight  is  adding  value  to  the  procurement  process,  the  statement  was 
revised  to  be  more  clear  and  direct. 

The  second  statement  revised  addressed  whether  time  spent  preparing  for  and 
participating  in  the  OSD  Pre- Award  Peer  Review  was  worthwhile,  utilizing  the  IPT’s 
time  efficiently  and  did  not  result  in  delays  to  the  acquisition.  The  statement  elicited  a 
strong  response  overall.  However,  because  it  captures  multiple  topics,  it  was  split  into 
two  separate  statements  to  prevent  confusion. 

Is  proper  training/mentorship  available  prior  to  Army  service  acquisition 

requirements/reviews  to  maximize  the  value  of  the  ASSP? 

The  statements  developed  to  address  this  question  focused  on  the  impacts  of  the 
Portfolio  Manager/Coordinator  on  the  following: 

•  Sufficiency  of  resources  provided  to  guide  the  Multi-Functional  IPT 

•  Clear  direction  of  resource  allocation  while  preparing  for  the  ASSP 

•  Value  added  feedback/input 

All  statements  addressing  this  question  elicited  strong  responses  and  remained 
unchanged. 

Are  Commands  capturing  metrics  to  document  the  requirements/reviews 

impacts? 

The  statements  developed  to  address  this  question  focused  on  the  Command’s  use 
of  metrics  to  document  and  assess  if: 

•  Additional  time  spent  impacted  procurement  timelines 

•  Reviews  have  led  to  cost  savings  and  reductions  in  services  contracting 
spending  rates 
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•  Reviews  have  led  to  an  increase  in  small  business  contracting 
opportunities 

•  Reviews  have  improved  requirements  definition 

Although  varied  responses  were  received,  field  test  results  indicate  that 
respondents  were  unsure  of  metrics  being  tracked.  Because  tracking  metrics  is  vital  to 
track  the  progress  and  impacts  of  the  additional  reviews,  the  statements  remained 
unchanged  and  responses  were  considered  impactful  as  it  may  indicate  that  metrics  are 
not  being  captured  or  if  captured,  not  shared  with  individual  programs.  Failure  to  share 
tracked  metrics  may  not  impress  importance  of  individual  focus  areas. 

Are  Commands  taking  appropriate  measures  to  capture  lessons  learned? 

The  statements  developed  to  address  this  question  focused  on  the  following: 

•  Documentation  and  dissemination  of  lessons  learned 

•  Use  of  lessons  learned  for  other  procurements 

•  Use  of  lessons  learned  for  ease  of  process  navigation  during  subsequent 
iterations 

•  Use  aggregate  data  collected  to  identify  improvements  to  service  acquisitions 

Similar  to  the  previous  question,  the  responses  from  these  statements  would  help 

to  track  if  lessons  learned  are  properly  captured  and  disseminated.  Field  test  results 
indicate  that  this  may  not  be  the  case. 

While  majority  of  the  statements  remain  unchanged,  one  statement  was  revised. 
Although  all  respondents  provided  feedback  to  whether  aggregate  data  collected  from 
post-award  Peer  Reviews  was  utilized  by  commands  to  identify  improvements  to  service 
acquisition,  the  statement  was  revised  to  elicit  a  stronger  response.  It  is  important  that 
commands  leverage  experiences  and  share  lessons  learned. 

The  results  from  the  field  test  led  to  minor  revisions  of  the  survey  instrument  and 
provided  insight  to  the  team  in  the  effectiveness  of  the  questions  and  whether  the 
responses  were  received  as  intended.  The  team  utilized  the  responses  to  improve  the 
statements  within  the  survey  instrument  where  needed. 
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VI.  SUMMARY  AND  RECOMMENDATIONS 


A.  SUMMARY 

This  JAP  examined  existing  OSD  and  Army  policies,  GAO  cases,  and  BBP 
initiatives  that  influence  the  Army  Service  acquisition  process.  The  in  depth  policy 
review  provided  insight  to  the  origins  on  the  Army  service  contracting  review 
requirements  that  are  currently  levied  on  higher  value  acquisitions.  The  review  helped 
highlight  key  policy  initiatives  that  the  service  review  process  is  geared  toward 
achieving. 

The  JAP  then  analyzed  all  of  these  policies  and  peripheral  influences  to  apply 
them  in  a  practical  setting.  This  process  also  helped  to  provide  a  better  understanding 
how  these  policies  align  and  interact.  The  analysis  pointed  to  areas  where  certain 
initiatives  might  contradict  each  other  and/or  lead  to  confusion  during  implementation. 

The  intention  of  the  JAP  is  to  analyze  the  Army  Service  Acquisition  Review 
Requirements  and  the  perceived  effectiveness  on  intended  improvements.  In  order  to 
accomplish  this,  the  team  developed  a  survey  instrument  to  help  assess  program 
participant’s  perceptions  on  whether  certain  targeted  goals  were  being  realized  or 
accomplished.  The  survey  was  then  field  tested  then  revised  to  reflect  inputs  from  a  small 
sample  size  of  respondents  who  represent  the  target  population.  The  final  survey 
instrument  is  provided  via  Appendix  2. 

B.  RECOMMENDATIONS 

Recommendation  1 

The  DASA(P)  SSM  should  consider  issuing  a  survey  to  program  personnel  that 
participate  in  the  Army  service  acquisition  review  process  with  questions  similar  to 
Appendix  2,  Survey  Questions  (to  include  discussion  with  the  Portfolio  Managers/PC, 
ASSP,  OSD  Peer  Reviews,  etc.). 

The  intent  of  this  project  was  to  analyze  the  perceived  effectiveness  of  the 
intended  improvements  of  the  Army  service  acquisition  review  process.  The  responses 
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received  during  the  field  test  reflect  that  many  of  the  intended  improvements  (such  as 
better  visibility,  additional  small  business  opportunities,  increased  use  of  incentive-based 
contracts,  etc.)  have  not  been  passed  down  to  the  acquisition  personnel,  as  one  may 
expect  it  would.  It  is  imperative  that  the  Army  receive  feedback  from  program 
participants  to  understand  whether  or  not  policy  goals  are  being  realized  or  achieved. 

Recommendation  2 

The  DASA(P)  SSM  should  assess  survey  feedback  and  evaluate  existing 
guidance/policies  to  ensure  resources  are  effectively  utilized. 

OSD  and  Army  policies,  as  well  as  many  BBP  influences  seek  to  improve  the 
overall  tradecraft  in  the  acquisition  of  services.  To  fulfill  many  of  the  intended 
improvements,  higher  level  oversight  and  more  stringent  review  processes  are  required 
for  Army  services  contracts.  BBP  also  aims  to  eliminate  unproductive  processes  and 
bureaucracy.  Feedback  from  the  survey  may  help  to  demonstrate  whether  or  not  certain 
processes  are  effective  in  achieving  policy  initiatives.  Unproductive  processes  should  be 
reviewed  with  the  intent  to  refine  or  eliminate  in  accordance  with  BBP. 

Compare  the  BBP  aim  to  eliminate  bureaucracy  and  improve  tradecraft  in  the 
acquisition  of  services.  This  information  should  ultimately  be  provided  to  OSD  for  their 
consideration  in  revising  policies  as  directed  from  their  level. 

Recommendation  3 

The  DASA(P)  SSM  should  ensure  lessons  learned  from  the  Army  service 
acquisition  review  process  are  captured  and  disseminated  to  requiring  activities  and 
contracting  agencies  to  maximize  value  add. 

The  DASA(P)  Office  of  the  SSM  currently  has  a  restricted  SharePoint  site  that 
provides  some  information  related  to  the  SAS;  however,  much  of  the  information  is 
outdated.  Program  participants  would  greatly  benefit  from  a  more  robust  library  of 
lessons  learned  and  strategy  recommendations  centralized  at  DASA(P).  DASA(P)  should 
mandate  that  lessons  learned  captured  from  each  individual  service  contract  review  are 
provided  for  inclusion  on  their  SharePoint  site  to  maximize  availability  and  exposure. 
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Recommendation  4 


The  Army  should  set  specific  guidelines  for  contracting  agencies  to  track  certain 
program  metrics  throughout  the  ASSP  process. 

The  survey  instrument  developed  during  the  JAP  specifically  intended  to  find  out 
whether  individual  commands  are  tracking  metrics  to  ensure  the  Army  service  acquisition 
review  process  is  helping  to  achieve  policy  initiatives  in  an  effective  and  efficient 
manner.  Survey  field  test  results  indicated  that  working-level  personnel  are  not  aware  that 
metrics  are  tracked.  The  Army  should  mandate  that  all  commands  track  metrics  to  assess 
how  the  service  review  processes  impacts  the  achievement  of  goals.  Commands  should 
also  track  the  additional  time  required  by  agency  level  personnel  to  complete  the  required 
ASSP  process  to  ensure  resource  inputs  exceed  outputs  and  provide  value  add. 

C.  CONCLUSIONS 

In  conclusion,  this  JAP  culminates  with  a  series  of  recommendations  for 
DASA(P)  SSM  and  Army  leadership  to  consider  to  help  maximize  the  value  of  the 
Army’s  service  review  requirements  process.  The  JAP  also  provides  a  completed  survey 
instrument  that  the  Army  could  utilize  to  help  assess  program  participant’s  perceptions  of 
the  process  in  an  anonymous  manner. 

We  believe  that  these  recommendations  could  help  to  improve  certain  aspects  of 
the  Army  services  requirements  reviews,  and  strongly  encourage  that  they  be  considered. 
We  also  believe  that  while  the  review  processes  do  help  to  improve  services  aspects  of 
service  acquisitions,  an  emphasis  should  be  placed  on  tracking  the  improvements  and 
weighing  the  value  against  the  resource  requirements. 
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APPENDIX  1.  SURVEY  ANALYSIS 


Append  -  Survey  Analysis 

Objeetive  1:  Are  the  Army 

serviee  aequisition  requirements/reviews  leading  to  desired  outeomes? 

Statement 

Rationale 

Field  Test  Response 

Assessment 

Increased  oversight  from  the 
offiee  of  the  Senior  Serviees 
Manager  (SSM)  has 
improved  visibility  into 
eommand  level  serviees 
eontraeting  spending. 


GAO  repeatedly  reported 
2011  Army  Services 
Implementation  Plan- 
Optimization  of  Army 
Serviees  Aequisition  strived 
to  achieve  better  visibility  into 
command  level  contract 
spending.  The  intention  of  the 
question  is  to  assess  whether 
or  not  there  is  a  perceived 
improvement. 


One  respondent  indieated  that  the 
statement  seemed  geared  toward 
the  organization  that  is  providing 
the  oversight  rather  than  that 
whieh  must  follow  the  review 
process.  Respondent  questioned 
merit  of  this  statement. 

Two  respondents  indieated  that 
their  response  to  the  statement 
would  be  neutral  as  written. 

One  respondent  indicated  that 
they  would  strongly  agree  with 
the  statement  as  written. 


The  field  test  suggests  that 
the  individual  respondents 
are  unsure  how  to  respond 
to  the  question.  Considering 
that  improved  visibility  into 
serviees  eontraet  spending 
was  a  primary  foeus  of  the 
2011  Army  policy,  it’s 
important  to  determine 
whether  or  not  this  is 
oecurring.  The  team  has 
decided  to  revise  statement 
to  be  elearer  and  elieit  a 
stronger  response. 

“Inereased  oversight  from 
the  offiee  of  the  Senior 
Serviees  Manager  (SSM) 
has  improved  your 
command’s  visibility  into 
their  serviees  eontraeting 
spending.” _ 
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Increased  oversight  through 
the  Army  service  acquisition 
requirements/reviews  has  led 
to  an  increase  in  small 
business  contracting 

opportunities. 

BBP  3.0  focus  to  promote 
competition  is  supplemented  by 
an  initiative  to  increase  small 
business  roles  and  opportunities. 
The  intent  of  the  questions  is  to 
determine  if  Army  requirements 
are  working  toward  this 
outcome. 

•  One  respondent  indicated  that 
the  statement  seemed  geared 
toward  the  organization  that  is 
providing  the  oversight  rather 
than  that  which  must  follow  the 
review  process.  Respondent 
questioned  merit  of  this 
statement. 

•  One  respondent  indicated  that 
they  would  disagree  with  the 
statement  as  written. 

•  One  respondent  indicated  that 
they  would  strongly  agree  with 
the  statement  as  written,  and 
noted  that  there  was  a  specific 
emphasis  placed  on  creating 
small  business  opportunities. 

•  One  respondent  indicated  that 
they  would  agree  with  the 
statement  as  written. 

Statement  currently  elicits 
strong  response  from 

participants.  No  change  is 
required. 

Increased  oversight  through 
the  Army  service  acquisition 
requirements/reviews  has  led 
to  an  increase  in  the 
utilization  of  incentive  based 

contracts. 

2011  Army  Services 

Implementation  Plan- 

Optimization  of  Army  Services 
Acquisition  strived  to  cost 
savings  by  implementing  more 
incentive  based  contracts.  The 

•  One  respondent  indicated  that 
the  statement  seemed  geared 
toward  the  organization  that  is 
providing  the  oversight  rather 
than  that  which  must  follow  the 
review  process.  Respondent 

The  field  test  suggests  that 
the  individual  respondents 
believe  this  statement  is  not 
applicable  to  their 

procurements.  Considering 
that  increasing  the 
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intention  of  the  question  is  to 
assess  whether  or  not  increased 
visibility  drove  higher  usage  of 
this  contract  type. 

questioned  merit  of  this 
statement. 

•  Two  respondents  indicated  that 
this  statement  is  not  applicable 
to  their  procurement. 

•  One  respondent  indicated  that 
they  would  agree  with  the 
statement  as  written. 

utilization  of  incentive 
based  contracts  was  a 
primary  focus  of  the  2011 
Army  policy,  it’s  important 
to  determine  whether  or  not 
this  is  occurring.  Responses 
indicate  that  this  objective 
may  not  be  understood  or  it 
is  not  being  achieved.  The 
team  has  decided  to  leave 
this  question  in  to  help 
provide  feedback  to  the 
Army  to  assess  whether  or 
not  the  review  process  is 
aiding  to  achieve  this  goal. 

Increased  oversight  through 
the  Army  service  acquisition 
requirements/reviews  has 

improved  requirements 

definition. 

BBP  3.0  focus  to  improve 
tradecraft  in  the  acquisition  of 
services  is  supplemented  by  an 
initiative  to  improve 

requirements  definition.  The 
intent  of  the  question  is  to 
determine  if  Army  requirements 
are  working  toward  this 
outcome. 

•  One  respondent  indicated  that 
the  statement  seemed  geared 
toward  the  organization  that  is 
providing  the  oversight  rather 
than  that  which  must  follow  the 
review  process.  Respondent 
questioned  merit  of  this 
statement. 

•  Two  respondents  indicated  that 
their  response  to  the  statement 
would  be  neutral  as  written. 

•  One  respondent  indicated  that 
they  would  agree  with  the 

The  field  test  suggests  that 
the  individual  respondents 
are  unsure  how  to  respond 
to  the  question.  Considering 
that  improving  requirements 
definition  was  a  primary 
focus  of  the  2011  Army 
policy,  it’s  important  to 
determine  whether  or  not 
this  is  occurring.  The  team 
has  decided  to  revise  the 
statement  to  be  clearer  and 
elicit  a  stronger  response. 
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statement  as  written. 

“Increased  oversight 

through  the  Army  service 
acquisition  requirements/ 
reviews  has  resulted  in 
feedback  that  has  improved 
the  requirements  package.” 

Increased  oversight  through 
the  Army  service  acquisition 
requirements/reviews  has  led 
to  an  increase  in  effective 
competition. 

2011  Army  Services 

Implementation  Plan- 

Optimization  of  Army  Services 
Acquisition  strived  to  achieve 
cost  savings  by  increasing 
competition  and  limiting  one  bid 
contracts.  The  intent  of  the 
question  is  to  assess  whether  or 
not  increased  visibility  drove 
higher  effective  competition. 

•  One  respondent  indicated  that 
the  statement  seemed  geared 
toward  the  organization  that  is 
providing  the  oversight  rather 
than  that  which  must  follow  the 
review  process.  Respondent 
questioned  merit  of  this 
statement. 

•  One  respondent  indicated  that 
their  response  to  the  statement 
would  be  neutral  as  written. 

•  Two  respondents  indicated  that 
they  would  agree  with  the 
statement  as  written. 

Although  one  respondent 
challenged  the  validity  of 
the  question,  and  one 
indicated  that  they  would 
respond  neutral  as  written, 
the  team  is  electing  to  keep 
the  question  as  is. 

Considering  that  increasing 
effective  competition  was  a 
primary  focus  of  the  2011 
Army  policy,  it’s  important 
to  determine  whether  or  not 
this  is  occurring.  Statement 
currently  elicits  strong 
response  from  half  of  the 
participants. 

Preparing  for  the  Army 
service  acquisition 

requirements/review  s 
requires  the  acquisition  team 
to  apply  more  critical 
thinking  toward  the 

acquisition  approach. 

BBP  3.0  focuses  on  improving 
the  tradecraft  of  the  acquisition 
of  services  workforce  as  well  as 
improving  the  professionalism 
of  the  total  acquisition 

workforce.  The  intent  of  this 
question  is  to  help  determine 

•  One  respondent  indicated 

concern  over  the  statement  as  it 
eludes  that  an  individual  does 
not  provide  critical  thinking 
unless  they  go  through  the 
process.  Respondent  further 
stated  that  people  might  not 

Although  one  respondent 
indicated  that  the  question 
may  be  off-putting,  the 
statement  received  a  strong 
response  from  the  remaining 
three  respondents.  The  team 
does  not  believe  that  the 
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whether  participation  in  review 
processes  are  driving  these 
changes. 

want  to  answer  statement  as 
written. 

•  Three  respondents  indicated 
that  they  would  agree  with  the 
statement  as  written. 

statement  implies  that 

critical  thinking  is  not 
otherwise  applied  under 
normal  acquisitions,  rather 
it  stresses  that  “more  critical 
thinking”  is  required.  No 
change  is  required. 

The  review  of  the  Service 
Acquisition  Strategy  through 
the  ASSP  helps  to  identify 
potential  risks  for  proposed 
acquisition  strategies. 

2010  Army  Regulation  70-13 
Management  and  Oversight  of 
Service  Acquisitions  established 
the  ASSP  to  allow  senior  Army 
functional  principals  to  review 
proposed  acquisition  strategies 
metrics  for  service  acquisitions 
to  reach  a  consensus  on  the 
strategies  that  are  most 
advantageous  to  the  Army.  The 
intent  of  the  question  is  to 
determine  if  stronger  acquisition 
strategies  are  resulting  from 
reviews. 

•  Three  respondents  indicated 
that  they  would  agree  with  the 
statement  as  written. 

Statement  currently  elicits 
strong  response  from 

participants.  No  change  is 
required. 

The  guidance  and  improved 
documents  provided  through 
the  Army  service  acquisition 
requirements/reviews  helped 
prevent  a  protest  from  being 
sustained. 

BBP  3.0  focuses  on  improving 
the  tradecraft  of  the  acquisition 
of  services  workforce.  The  intent 
of  this  question  is  to  help 
determine  if  review  process  is 
leading  toward  improvements  in 
services  acquisitions, 

specifically  with  respect  to 

•  Two  respondents  indicated  that 
this  statement  was  not 
applicable  to  their 
procurements.  One  of  these 
respondents  stated  that  they  are 
not  sure  what  is  meant  by 
“improved  documentation 

The  statement  as  written  did 
not  response  strong 

responses.  The  respondents 
were  confused  or  unaware 
of  improved  documentation 
provided  throughout  the 
review  process.  The  true 
intent  of  the  statement  is 
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protest  prevention. 

provided.” 

•  One  respondent  indicated  that 
their  response  to  the  statement 
would  be  neutral  as  written, 
and  stated  they  are  “unaware  of 
any  improved  documents.” 

rather  to  determine  if  the 
process  helped  prevent  a 
protest  from  being 

sustained.  The  team  elected 
to  revise  the  statement  to 
elicit  a  stronger  response. 

“The  guidance  provided 
through  the  Army  service 
acquisition  requirements/ 
reviews  helped  prevent  a 
protest  from  being 

sustained.” 

Objective  2:  Does  the  value  ac 
the  process? 

ded  through  completion  of  the  requirements/reviews  exceed  the  resource  outputs  required  to  complete 

Statement 

Rationale 

Field  Test  Response 

Assessment 

Increased  oversight  through 
the  Army  service  acquisition 
requirements/reviews 
provides  critical  feedback  for 
acquisition  team. 

BBP  3.0  focuses  on  improving 
the  tradecraft  of  the  acquisition 
of  services  workforce  as  well  as 
improving  the  professionalism 
of  the  total  acquisition 

workforce.  The  intent  of  this 
question  is  to  help  determine 
whether  participation  in  review 
processes  drive  these  changes. 

•  Three  respondents  indicated 
that  they  would  agree  with  the 
statement  as  written. 

•  One  respondent  indicated  that 
feedback  from  the  ASSP 
review  resulted  in  changes  to 
the  acquisition  prior  to 
proceeding  to  the  OSD-level 
peer  review. 

Statement  currently  elicits 
strong  response  from 

participants.  No  change  is 
required. 
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Results  from  the  ASSP 
review  process  have  led  to 
significant  improvements  in 
acquisition  approach. 

2010  Army  Regulation  70-13 
Management  and  Oversight  of 
Service  Acquisitions  established 
the  ASSP  to  allow  senior  Army 
functional  principals  to  review 
proposed  acquisition  strategies 
metrics  for  service  acquisitions 
to  reach  a  consensus  on  the 
strategies  that  are  most 
advantageous  to  the  Army.  The 
intent  of  the  question  is  to 
determine  if  stronger  acquisition 
strategies  are  resulting  from 
reviews. 

•  One  respondent  indicated  that 
their  response  to  the  statement 
would  be  neutral  as  written. 

•  Two  respondents  indicated  that 
they  would  agree  with  the 
statement  as  written. 

Overall,  statement  currently 
elicits  strong  response  from 
participants.  Even  though 
there  is  one  neutral 
response,  the  statement 
should  remain  to  help 
determine  the  perceived 
value  add  of  the  process  on 
the  overall  procurement.  No 
change  is  required. 

Time  spent  preparing  for  and 
participating  in  the  ASSP 
review  process  improves 
skillset  of  acquisition  team. 

BBP  3.0  focuses  on  improving 
the  tradecraft  of  the  acquisition 
of  services  workforce  as  well  as 
improving  the  professionalism 
of  the  total  acquisition 

workforce.  The  intent  of  this 
question  is  to  help  determine 
whether  participation  in  review 
processes  are  driving  these 
changes. 

•  One  respondent  indicated  that 
they  would  strongly  agree  with 
the  statement  as  written. 

•  Two  respondents  indicated  that 
they  would  agree  with  the 
statement  as  written. 

Statement  currently  elicits 
strong  response  from 

participants.  No  change  is 
required. 
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Time  spent  preparing  for  and 
participating  in  the  ASSP 
review  process  places 

schedule  risk  on  programs. 

2010  Army  Regulation  70-13 
Management  and  Oversight  of 
Service  Acquisitions  established 
the  ASSP  to  allow  senior  Army 
functional  principals  to  review 
proposed  acquisition  strategies 
metrics  for  service  acquisitions 
to  reach  a  consensus  on  the 
strategies  that  are  most 
advantageous  to  the  Army.  BBP 
3.0  focuses  on  eliminating 
unproductive  processes  or 
bureaucracy.  The  intent  of  the 
question  is  to  determine  if 
review  process  places  schedule 
risk  on  programs. 

•  One  respondent  indicated  that 
their  response  would  be  neutral 
as  written. 

•  One  respondent  indicated  that 
they  would  strongly  agree  with 
the  statement  as  written. 

•  One  respondent  indicated  that 
they  would  disagree  with  the 
statement  as  written,  and  stated 
that  the  review  process  placed 
no  schedule  risk  on  their 
program. 

Overall,  statement  currently 
elicits  strong  response  from 
participants.  Even  though 
there  is  one  neutral 
response,  the  statement 
should  remain  to  help 
determine  the  perception  of 
the  schedule  risk  placed  on 
the  procurement.  No  change 
is  required. 

Utilizing  the  Seven  Steps  to 
Service  Acquisition 

Processes  demonstrated 

during  Services  Acquisition 
Workshop  improved 

procurement  outcomes. 

DPAP  memorandum,  subject: 
Service  Acquisition  Workshop 
dated  6  Dec  2012  mandates  that 
Service  Acquisition  Workshop 
training  be  completed  prior  to 
submitting  acquisition  strategies 
for  services  contracts  valued 
higher  than  $100m.  The  intent  of 
the  question  is  to  determine  if 
participation  is  driving 

acquisition  improvements. 

•  One  respondent  indicated  that 
the  statement  was  not 
applicable  to  their 
procurement,  and  stated  that 
DPAP  waived  the  requirement 
for  a  SAW  on  their  program. 

•  One  respondent  indicated  that 
they  would  agree  with  the 
statement  as  written. 

•  One  respondent  indicated  that 
they  would  disagree  with  the 
statement  as  written,  and  stated 

Overall,  statement  currently 
elicits  strong  response  from 
participants.  Even  though 
there  is  not  applicable 
response,  the  statement 
should  remain  to  help 
determine  the  perception  of 
the  value  add  of  the 
Services  Acquisition 

Workshop.  No  change  is 
required. 
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that  they  did  not  see  the  benefit 
of  the  SAW  after  participating. 

Using  the  Acquisition 

Requirements  Roadmap  Tool 
(ARRT)  assisted  the 

requiring  activity  to  ensure 
Performance  Based 

requirements  were  utilized 
for  the  acquisition. 

BBP  3.0  focus  to  improve 
tradecraft  in  the  acquisition  of 
services  is  supplemented  by  an 
initiative  to  improve 

requirements  definition.  The 
intention  of  the  question  is  to 
assess  whether  use  of  ARRT  to 
create  performance  based 

requirements  is  utilized  to  help 
achieve  this  goal. 

•  One  respondent  indicated  that 
the  statement  was  not 
applicable  to  their 
procurement,  and  stated  ARRT 
was  not  used  for  their  program. 

•  One  respondent  indicated  that 
their  response  would  be  neutral 
as  written. 

•  One  respondent  indicated  that 
they  would  disagree  with  the 
statement  as  written,  and  stated 
that  the  ARRT  tool  is  not 
locally  endorsed  or  mandated. 

The  responses  received 
indicate  that  ARRT  is  not 
being  utilized  wide  scale. 
The  team  has  decided  to 
keep  the  statement  to 
provide  insight  to  Army 
leadership  on  how  many 
programs  are  leveraging  the 
tool.  Feedback  may  suggest 
that  tool  is  not  being 
implemented  and  the  Army 
could  consider  pushing  the 
initiative  further.  The  DOD 
PGI  says  “should”  use 
ARRT,  which  could  be 
changed  to  “shall”  to 
increase  usage. 

Your  command  had 

sufficient  personnel  to 
participate  in  the  additional 
Army  service  acquisition 
requirements/reviews 
without  causing  delay  in 
other  activities 

BBP  3.0  focus  on  eliminating 
unproductive  processes  or 
bureaucracy  is  supplemented  by 
an  initiative  to  reduce  cycle 
times  while  ensuring  sound 
investment  decisions.  The 

intention  of  the  question  is  to 
assess  sufficient  resources  are 
available  to  achieve  this  goal. 

•  One  respondent  indicated  that 
they  would  strongly  disagree 
with  the  statement  as  written. 

•  Two  respondents  indicated  that 
they  would  agree  with  the 
statement  as  written. 

Statement  currently  elicits 
strong  response  from 

participants.  No  change  is 
required. 
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Time  spent  preparing  for  and 
participating  in  the  OSD  Pre- 
Award  Peer  Review 

provided  valuable  input  to 
the  acquisition  resulting  in 
improvements  to  the 

Acquisition  Strategy. 

USD  (AT&L)  memorandum, 
subject:  Peer  Reviews  of 

Contracts  for  Supplies  and 
Services  dated  29  Sept  2008 
established  requirements  for  pre¬ 
award  peer  reviews  for  services 
contract  value  higher  than  $1B. 
The  intent  of  the  question  is  to 
determine  if  stronger  acquisition 
strategies  are  resulting  from 
reviews. 

•  One  respondent  indicated  that 
the  statement  was  not 
applicable  to  their 
procurement. 

•  One  respondent  indicated  that 
their  response  would  be  neutral 
as  written,  and  stated  that  OSD 
did  not  change  their  strategy. 

•  One  respondent  indicated  that 
they  would  strongly  agree  with 
the  statement  as  written. 

The  statement  did  not  elicit 
strong  responses  as  written. 
The  team  has  decided  to 
revise  the  statement  to  make 
it  clearer  and  more  direct.  It 
is  important  to  determine 
whether  the  additional  OSD 
level  oversight  is  adding 
value  to  the  procurement 
process. 

Note:  Post  award  peer 
review  teams  may  differ 
from  the  team  that  initially 
awarded  the  contract.  An  N/ 
A  response  would  be 
appropriate  in  this  instance. 

“Time  spent  preparing  for 
and  participating  in  the 
OSD  Pre-Award  Peer 
Review  resulted  in 

improvements  to  the 

Acquisition  Strategy.” 
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Time  spent  preparing  for  and 
participating  in  the  OSD  Pre- 
Award  Peer  Review  was 
worthwhile,  utilizing  the 
IPTs  time  efficiently  and  did 
not  result  in  delays  to  the 
acquisition 

USD  (AT&L)  memorandum, 
subject:  Peer  Reviews  of 

Contracts  for  Supplies  and 
Services  dated  29  Sept  2008 
established  requirements  for  pre¬ 
award  peer  reviews  for  services 
contract  value  higher  than  $1B. 
BBP  3.0  focuses  on  eliminating 
unproductive  processes  or 
bureaucracy.  The  intent  of  the 
question  is  to  determine  if 
review  process  was  productive 
and  efficient. 

•  One  respondent  indicated  that 
the  statement  was  not 
applicable  to  their 
procurement. 

•  One  respondent  indicated  that 
they  would  strongly  agree  with 
the  statement  as  written,  and 
stated  that  their  program  has  a 
positive  OSD  review 
experience. 

•  One  respondent  indicated  that 
they  would  agree  with  the 
statement  as  written.  The 
respondent  stated  that  while  the 
process  was  worthwhile  to  gain 
OSD  perspective.  The  process 
added  significant  time  to  the 
milestone  schedule. 

Although  the  statement 
elicited  a  strong  response 
overall,  the  team  realized 
that  there  are  multiple  topics 
of  focus  under  this 
statement.  The  team  elected 
to  revise  the  statement  in 
split  out  into  two  separate 
items  to  prevent  potential 
confusion  going  forward.  . 

“The  IPT  utilized  time 
efficiently  while  preparing 
for  the  OSD  Pre-Award 
Peer  Review.  “ 

“Time  spent  preparing  for 
the  OSD  Pre-Award  Peer 
Review  resulted  in  delays  to 
the  acquisition.” 

Time  spent  preparing  for  and 
participating  in  the  OSD 
Post-Award  Peer  Review 
added  value  to  the  services 
acquisition  process. 

USD  (AT&L)  memorandum, 
subject:  Peer  Reviews  of 

Contracts  for  Supplies  and 
Services  dated  29  Sept  2008 
established  requirements  for  pre¬ 
award  peer  reviews  for  services 
contract  value  higher  than  $1B. 
The  intent  of  the  question  is  to 
determine  if  time  spent 

•  Two  respondents  indicated  that 
the  statement  was  not 
applicable  to  their 
procurement. 

•  One  respondent  indicated  that 
they  would  agree  with  the 
statement  as  written. 

Although  two  respondents 
indicated  that  the  statement 
was  not  applicable  to  their 
procurement,  the  team  has 
elected  to  keep  the 
statement  as  is.  It  is  possible 
that  these  individuals  have 
not  yet  been  through  the 
peer  review  process.  It’s 
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preparing  for  review  process 
was  perceived  as  productive. 

important  to  assess  the 
perceived  value  add  of 
higher  level  OSD  reviews. 

Participating  in  an  annual 
OSD  Post-Award  Peer 
Review  has  proven  valuable 
by  providing  guidance  to 
improve  oversight,  best 
practices  and  lessons  learned. 

USD  (AT&L)  memorandum, 
subject:  Peer  Reviews  of 

Contracts  for  Supplies  and 
Services  dated  29  Sept  2008 
established  requirements  for  pre¬ 
award  peer  reviews  for  services 
contract  value  higher  than  $1B. 
The  intent  of  the  question  is  to 
determine  if  time  spent 
preparing  for  review  process 
was  perceived  as  productive. 

•  Three  respondents  indicated 
that  the  statement  was  not 
applicable  to  their 
procurement. 

Although  three  respondents 
indicated  that  the  statement 
was  not  applicable  to  their 
procurement,  the  team  has 
elected  to  keep  the 
statement  as  is.  It  is  possible 
that  these  individuals  have 
not  yet  been  through  the 
peer  review  process.  It’s 
important  to  assess  the 
perceived  value  add  of 
higher  level  OSD  reviews. 

Note:  Post  award  peer 
review  teams  may  differ 
from  the  team  that  initially 
awarded  the  contract.  An  N/ 
A  response  would  be 
appropriate  in  this  instance. 

Objective  3:  Is  proper  training/mentorship  available  prior  to  the  Army  service  acquisition  requirements/reviews  to  maximize  the 
value  of  the  ASSP? 

Statement 

Rationale 

Field  Test  Response 

Assessment 

The  Portfolio  Manager/ 
Coordinator  provides 

sufficient  resources  to  guide 
the  Multi-  Functional  IPT 

2011  Army  Services 

Implementation  Plan- 

Optimization  of  Army  Services 
Acquisition  established  a 

•  One  respondent  indicated  that 
they  would  strongly  disagree 
with  the  statement  as  written, 
and  noted  that  the  portfolio 

Statement  currently  elicits 
strong  response  from 

participants.  No  change  is 
required. 
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through  ASSP  preparation. 

portfolio  management  approach 
to  guide  acquisitions.  The  intent 
of  the  question  is  to  determine  is 
support  level  provided  is 
sufficient. 

manager  was  unable  to  offer 
any  insight. 

•  Two  respondents  indicated  that 
they  would  agree  with  the 
statement  as  written. 

The  Portfolio  Manager/ 
Coordinator  worked  with 
acquisition  team  during 
procurement  planning  and 
provided  valuable  feedback/ 
input. 

2011  Army  Services 

Implementation  Plan- 

Optimization  of  Army  Services 
Acquisition  established  a 

portfolio  management  approach 
to  guide  acquisitions.  BBP  3.0 
focuses  on  eliminating 

unproductive  processes  or 
bureaucracy.  The  intent  of  the 
question  is  to  determine 
feedback  is  value  added. 

•  Two  respondents  indicated  that 
they  would  agree  with  the 
statement  as  written. 

•  One  respondent  indicated  that 
they  would  disagree  with  the 
statement  as  written. 

Statement  currently  elicits 
strong  response  from 

participants.  No  change  is 
required. 

The  Portfolio  Manager/ 
Coordinator  provides  clear 
direction  of  where  best  to 
focus  resources  while 

preparing  for  the  ASSP. 

2011  Army  Services 

Implementation  Plan- 

Optimization  of  Army  Services 
Acquisition  established  a 

portfolio  management  approach 
to  guide  acquisitions.  BBP  3.0 
focuses  on  eliminating 

unproductive  processes  or 
bureaucracy.  The  intent  of  the 
question  is  to  determine  if 
direction  provided  is  efficient 
and  productive. 

•  Two  respondents  indicated  that 
they  would  agree  with  the 
statement  as  written. 

•  One  respondent  indicated  that 
they  would  disagree  with  the 
statement  as  written. 

Statement  currently  elicits 
strong  response  from 

participants.  No  change  is 
required. 
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The  current  acquisition  team 
contains  personnel  who  have 
previously  participated  in  the 
ASSP  review  process. 

BBP  3.0  focus  to  improve  the 
professionalism  of  the 

acquisition  workforce  is 

supplemented  by  an  initiative  to 
establish  higher  standard  for  key 
leadership  positions,  which 
includes  relevant  experience. 
The  intent  of  this  question  is  to 
determine  if  key  acquisition 
leaders  have  this  specific 
relevant  experience. 

•  One  respondent  indicated  that 
their  response  would  be  neutral 
as  written. 

•  One  respondent  indicated  that 
they  would  agree  with  the 
statement  as  written. 

•  One  respondent  indicated  that 
they  would  strongly  disagree 
with  the  statement  as  written, 
and  stated  that  while  their  team 
was  “green”  still  successful. 

The  statement  currently 
elicits  strong  response  from 
participants.  No  change  is 
required. 

Objective  4:  Are  Commands  capturing  metrics  to  document  the  requirements/reviews  impacts? 

Statement 

Rationale 

Field  Test  Response 

Assessment 

Your  command  tracks 

metrics  to  document 

additional  time  spent 

preparing  for  the  Army 
service  acquisition 

requirements/reviews  to 

understand  impacts  on 
procurement  timelines? 

BBP  3.0  focus  on  eliminating 
unproductive  processes  or 
bureaucracy  is  supplemented  by 
an  initiative  to  reduce  cycle 
times  while  ensuring  sound 
investment  decisions.  The 

intention  of  the  question  is  to 
assess  whether  or  commands  are 
tracking  progress  toward  this 
goal. 

•  One  respondent  indicated  that 
the  statement  was  not 
applicable  to  their 
procurement,  and  stated  that 
they  are  unaware  if  this  is 
being  tracked. 

•  One  respondent  indicated  that 
their  response  would  be  neutral 
as  written,  and  stated  that  they 
are  unaware  what  metrics  are 
being  tracked. 

Although  two  respondents 
indicate  that  they  are  not 
sure  what  metrics  are  being 
tracked,  the  team  has 
elected  to  keep  the  question 
as  is.  The  result  of  the 
survey  may  indicate  that 
metrics  are  not  being 
captured  or  they’re  not 
being  shared  with  individual 
programs.  Failure  to  share 
tracked  metrics  may  not 
impress  importance  of 
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•  One  respondent  indicated  that 
they  would  agree  with  the 
statement  as  written. 

individual  focus  areas. 

Your  command  tracks 

metrics  to  assess  whether  the 
Army  service  acquisition 
requirements/reviews  has  led 
to  cost  savings  and 

reductions  in  services 

contracting  spend  rates. 

2011  Army  Services 

Implementation  Plan- 

Optimization  of  Army  Services 
Acquisition  strived  to  achieve 
better  visibility  into  command 
level  contract  spending.  The 
intention  of  the  question  is  to 
assess  whether  or  commands  are 
tracking  progress  toward  this 
goal. 

•  One  respondent  indicated  that 
the  statement  was  not 
applicable  to  their 
procurement,  and  stated  that 
they  are  unaware  if  this  is 
being  tracked. 

•  One  respondent  indicated  that 
their  response  would  be  neutral 
as  written,  and  stated  that  they 
are  unaware  what  metrics  are 
being  tracked. 

•  One  respondent  indicated  that 
they  would  agree  with  the 
statement  as  written. 

Although  two  respondents 
indicate  that  they  are  not 
sure  what  metrics  are  being 
tracked,  the  team  has 
elected  to  keep  the  question 
as  is.  The  result  of  the 
survey  may  indicate  that 
metrics  are  not  being 
captured  or  they’re  not 
being  shared  with  individual 
programs.  Failure  to  share 
tracked  metrics  may  not 
impress  importance  of 
individual  focus  areas. 

Your  command  tracks 

metrics  to  assess  whether  the 
Army  service  acquisition 
requirements/reviews  has  led 
to  an  increase  in  small 
business  contracting 

opportunities. 

BBP  3.0  focus  to  promote 
competition  is  supplemented  by 
an  initiative  to  increase  small 
business  roles  and  opportunities. 
The  intention  of  the  question  is 
to  assess  whether  or  commands 
are  tracking  progress  toward  this 
goal. 

•  One  respondent  indicated  that 
the  statement  was  not 
applicable  to  their 
procurement,  and  stated  that 
they  are  unaware  if  this  is 
being  tracked. 

•  One  respondent  indicated  that 
their  response  would  be  neutral 

Although  a  respondent 
indicated  that  they  are  not 
sure  what  metrics  are  being 
tracked,  the  team  has 
elected  to  keep  the  question 
as  is.  The  result  of  the 
survey  may  indicate  that 
metrics  are  not  being 
captured  or  they’re  not 
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as  written. 

•  One  respondent  indicated  that 
they  would  agree  with  the 
statement  as  written. 

being  shared  with  individual 
programs.  Failure  to  share 
tracked  metrics  may  not 
impress  importance  of 
individual  focus  areas. 

Your  command  tracks 

metrics  to  assess  whether  the 
Army  service  acquisition 
requirements/reviews  has 

improved  requirements 

definition? 

BBP  3.0  focus  to  improve 
tradecraft  in  the  acquisition  of 
services  is  supplemented  by  an 
initiative  to  improve 

requirements  definition.  The 
intention  of  the  question  is  to 
assess  whether  or  commands  are 
tracking  progress  toward  this 
goal. 

•  One  respondent  indicated  that 
the  statement  was  not 
applicable  to  their 
procurement,  and  stated  that 
they  are  unaware  if  this  is 
being  tracked. 

•  One  respondent  indicated  that 
their  response  would  be  neutral 
as  written. 

•  One  respondent  indicated  that 
they  would  agree  with  the 
statement  as  written. 

Although  a  respondent 
indicated  that  they  are  not 
sure  what  metrics  are  being 
tracked,  the  team  has 
elected  to  keep  the  question 
as  is.  The  result  of  the 
survey  may  indicate  that 
metrics  are  not  being 
captured  or  they’re  not 
being  shared  with  individual 
programs.  Failure  to  share 
tracked  metrics  may  not 
impress  importance  of 
individual  focus  areas. 
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Objective  5:  Are  commands  taking  appropriate  measures  to  capture  lessons  learned? 

Statement 

Rationale 

Field  Test  Response 

Assessment 

Your  command  documents  and 
disseminates  lessons  learned 
from  the  Army  service 
acquisition  requirements/ 

reviews. 

BBP  3.0  focuses  on  improving  the 
tradecraft  of  the  acquisition  of 
services  workforce  as  well  as 
improving  the  professionalism  of 
the  total  acquisition  workforce.  The 
intent  of  this  question  is  to  help 
determine  if  commands  are 
applying  lessons  learned  toward 
other  procurements  and  driving 
these  changes. 

•  Two  respondents  indicated  that 
their  response  would  he  neutral  as 
written.  One  of  these  respondents 
stated  that  lessons  learned  from 
their  program  were  offered  to 

OSD,  hut  they  are  unaware  of 
what  happened  to  them.  They  also 
noted  that  they  leveraged  lessons 
learned  from  OSD  wehsite  prior 
to  peer  review. 

•  One  respondent  indicated  that 
they  would  agree  with  the 
statement  as  written. 

Two  neutral  responses  indicate 
that  lessons  learned  are  not 
being  properly  captured  and 
disseminated.  The  team  has 
elected  to  keep  the  statement 
as  is.  It  is  important  that 
commands  leverage 

experiences  and  share  lessons 
learned.  The  survey  may 
provide  insight  that  this  is  not 
being  done  wide  scale. 

Lessons  learned  from  the  Army 
service  acquisition 

requirements/reviews  are 

applied  to  other  procurements, 
to  include  those  service 
acquisitions  that  are  not  held  to 
the  ASSP  requirements  (under 
$250M). 

BBP  3.0  focuses  on  improving  the 
tradecraft  of  the  acquisition  of 
services  workforce  as  well  as 
improving  the  professionalism  of 
the  total  acquisition  workforce.  The 
intent  of  this  question  is  to  help 
determine  commands  are  applying 
lessons  learned  toward  other 
procurements  and  driving  these 
changes. 

•  One  respondent  indicated  that 
their  response  would  he  neutral  as 
written. 

•  One  respondent  indicated  that 
they  would  strongly  agree  with 
the  statement  as  written. 

•  One  respondent  indicated  that 
they  would  agree  with  the 
statement  as  written. 

The  statement  currently  elicits 
strong  response  from 

participants.  No  change  is 
required. 
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Utilizing  lessons  learned  from 
the  Army  service  acquisition 
requirements/reviews  make  it 
easier  to  navigate  process 
during  subsequent  iterations. 

BBP  3.0  focuses  on  improving  the 
tradecraft  of  the  acquisition  of 
services  workforce.  The  intent  of 
this  question  is  to  help  determine 
lessons  learned  are  leading  toward 
long  term  service  improvements. 

•  One  respondent  indicated  that 
they  would  strongly  agree  with 
the  statement  as  written. 

•  Two  respondents  indicated  that 
they  would  agree  with  the 
statement  as  written. 

Statement  currently  elicits 
strong  response  from 

participants.  No  change  is 
required. 

Aggregate  data  collected  from 
post-award  Peer  Reviews  is 
utilized  by  your  command  to 
identify  improvements  to 
service  acquisition. 

BBP  3.0  focuses  on  improving  the 
tradecraft  of  the  acquisition  of 
services  workforce.  The  intent  of 
this  question  is  to  help  determine  if 
commands  are  utilizing  all  data 
collected  to  improve  service 
acquisitions. 

•  Two  respondents  indicated  that 
the  statement  was  not  applicable 
to  their  procurement. 

•  One  respondent  indicated  that 
they  would  agree  with  the 
statement  as  written. 

Two  respondents  indicated 
that  the  statement  was  not 
applicable  to  their 

procurement.  It  is  possible  that 
these  individuals  have  not  yet 
been  through  the  peer  review 
process.  It’s  important  to 
assess  the  perceived  value  add 
of  higher  level  OSD  reviews. 
The  team  has  elected  to  revise 
the  question  to  elicit  a  stronger 
response. 

Note:  Post  award  peer  review 
teams  may  differ  from  the 
team  that  initially  awarded  the 
contract.  An  N/A  response 
would  be  appropriate  in  this 
instance. 

“Lessons  learned  from  post¬ 
award  Peer  Reviews  are 
utilized  by  your  command  to 
identify  improvements  to 
service  acquisition.” 
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APPENDIX  2.  ARMY  SERVICE  ACQUISITION  REVIEW 

ASSESSMENT 


Statement 

Strongly 

Agree 

Agree 

Neutral 

Disagree 

Strongly 

Disagree 

N/A 

Increased  oversight  from  the  office  of 
the  Senior  Services  Manager  (SSM)  has 
improved  your  command’s  visibility  into 
their  services  contracting  spending 

Comments: 

Increased  oversight  through  the  Army 
service  acquisition  requirements/ 

reviews  has  led  to  an  increase  in  small 
business  contracting  opportunities. 

Comments: 

Increased  oversight  through  the  Army 
service  acquisition  requirements/ 

reviews  has  led  to  an  increase  in  the 
utiLi2ation  of  incentive  based  contracts. 

Comments: 

Increased  oversight  through  the  Army 
service  acquisition  requirements/ 

reviews  has  resulted  in  feedback  that  has 
improved  the  requirements  package. 

Comments: 

Increased  oversight  through  the  Army 
service  acquisition  requirements/ 

reviews  has  led  to  an  increase  in 
effective  competition. 

Comments: 

Preparing  for  the  Army  service 
acquisition  requirements /reviews 

requires  the  acquisition  team  to  apply 
more  critical  thinking  toward  the 
acquisition  approach. 

Comments: 

The  review  of  the  Service  Acquisition 
Strategy  through  the  ASSP  helps  to 
identify  potential  risks  for  proposed 
acquisition  strategies. 

Comments: 

The  guidance  provided  through  the 
Army  service  acquisition  requirements/ 
reviews  helped  prevent  a  protest  from 
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Statement 

Strongly 

Agree 

Agree 

Neutral 

Disagree 

Strongly 

Disagree 

N/A 

being  sustained.” 

Comments; 

The  guest  staff  was  knowledgeable  and 
fully  answered  my  questions  about  the 
area. 

Comments: 

Increased  oversight  through  the  Army 
service  acquisition  requirements/ 

reviews  provides  critical  feedback  for 
acquisition  team. 

Comments; 

Results  from  the  ASSP  review  process 
have  led  to  significant  improvements  in 
acquisition  approach. 

Comments: 

Time  spent  preparing  for  and 
participating  in  the  ASSP  review  process 
improves  skiUset  of  acquisition  team. 

Comments; 

Time  spent  preparing  for  and 
participating  in  the  ASSP  review  process 
places  schedule  risk  on  programs. 

Comments: 

Utili2ing  the  Seven  Steps  to  Service 
Acquisition  Processes  demonstrated 
during  Services  Acquisition  Workshop 
improved  procurement  outcomes. 

Comments: 

Using  the  Acquisition  Requirements 
Roadmap  Tool  (ARRT)  assisted  the 
requiring  activity  to  ensure  Performance 
Based  requirements  were  utili2ed  for  the 
acquisition. 

Comments: 

Your  command  had  sufficient  personnel 
to  participate  in  the  additional  Army 
service  acquisition  requirements/ 

reviews  without  causing  delay  in  other 
activities 

Comments: 

Time  spent  preparing  for  and 
participating  in  the  OSD  Pre-Award 
Peer  Review  resulted  in  improvements 
to  the  Acquisition  Strategy. 
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Statement 

Strongly 

Agree 

Agree 

Neutral 

Disagree 

Strongly 

Disagree 

N/A 

Comments: 

The  IPT  utili2ed  time  efficiently  while 
preparing  for  the  OSD  Pre-Award  Peer 
Review. 

Comments: 

Time  spent  preparing  for  the  OSD  Pre- 
Award  Peer  Review  resulted  in  delays  to 
the  acquisition. 

Comments: 

Time  spent  preparing  for  and 
participating  in  the  OSD  Post-Award 
Peer  Review  added  value  to  the  services 
acquisition  process. 

Comments: 

Participating  in  an  annual  OSD  Post- 
Award  Peer  Review  has  proven  valuable 
by  providing  guidance  to  improve 
oversight,  best  practices  and  lessons 
learned. 

Comments: 

The  Portfolio  Manager/ Coordinator 
provides  sufficient  resources  to  guide 
the  Multi-  Functional  IPT  through 
ASSP  preparation. 

Comments: 

The  Portfolio  Manager/ Coordinator 
worked  with  acquisition  team  during 
procurement  planning  and  provided 
valuable  feedback/input. 

Comments: 

The  Portfolio  Manager/ Coordinator 
provides  clear  direction  of  where  best  to 
focus  resources  while  preparing  for  the 
ASSP. 

Comments: 

The  current  acquisition  team  contains 
personnel  who  have  previously 
participated  in  the  ASSP  review  process. 

Comments: 

Your  command  tracks  metrics  to 
document  additional  time  spent 
preparing  for  the  Army  service 
acquisition  requirements/ reviews  to 
understand  impacts  on  procurement 
timelines? 
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Statement 

Strongly 

Agree 

Agree 

Neutral 

Disagree 

Strongly 

Disagree 

N/A 

Comments; 

Your  command  tracks  metrics  to  assess 
whether  the  Army  service  acquisition 
requirements/reviews  has  led  to  cost 
savings  and  reductions  in  services 
contracting  spend  rates. 

Comments: 

Your  command  tracks  metrics  to  assess 
whether  the  Army  service  acquisition 
requirements /reviews  has  led  to  an 
increase  in  small  business  contracting 
opportunities. 

Comments; 

Your  command  tracks  metrics  to  assess 
whether  the  Army  service  acquisition 
requirements/reviews  has  improved 
requirements  definition? 

Comments: 

Your  command  documents  and 
disseminates  lessons  learned  from  the 
Army  service  acquisition  requirements/ 
reviews 

Comments; 

Lessons  learned  from  the  Army  service 
acquisition  requirements /reviews  are 
applied  to  other  procurements,  to 
include  those  service  acquisitions  that 
are  not  held  to  the  ASSP  requirements 
(under  |250M). 

Comments: 

Utili2ing  lessons  learned  from  the  Army 
service  acquisition  requirements/ 

reviews  make  it  easier  to  navigate 
process  during  subsequent  iterations. 

Comments; 

Lessons  learned  from  Post-Award  peer 
reviews  are  utili2ed  by  your  command  to 
identify  improvements  to  service 
acquisition. 

Comments; 
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Statement 


Strongly 

Agree 

Agree 

Neutral 

Disagree 

Strongly 

Disagree 

N/A 


Additional  Comments: 
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APPENDIX  3.  SURVEY  INSTRUCTIONS 


Our  thesis  examines  the  existing  policies/guidance  for  service  acquisitions  (OSD,  Army, 
GAO  cases  and  BBP),  analyzes  the  policies/guidance  for  service  acquisitions,  provides 
the  research  methodology  to  help  determine  the  survey  questions  and  will  conclude  with 
the  findings  and  results.  This  survey  “field  test”  we  have  provided  fits  into  the  research 
methodology  where  we  describe  the  survey  development,  field  testing  procedure  and  will 
analyze  the  provided  results. 

These  service  acquisition  review  requirements  are  relatively  new  and  the  objective  of  our 
analysis  is  to  create  a  survey  that  could  be  issued  at  the  conclusion  of  the  ASSP  process 
to  obtain  input  from  the  commands.  The  field  test  of  the  survey  is  designed  to  determine 
the  appropriateness  of  the  question,  if  it  is  clear/easy  to  understand,  if  questions  are  free 
of  bias,  and  whether  you  are  comfortable  responding  to  it.  Your  input  is  very  crucial  for 
the  success  of  our  thesis. 

The  use  of  Army  service  acquisition  requirements/reviews  in  the  survey  is  inclusive  of 
the  entire  process  of  coordinating  with  the  Portfolio  Managers,  Portfolio  Coordinators 
and  Senior  Service  Manager.  This  includes  participation  in  the  SAW,  completion  of  a 
BCA  and  CBA  (if  applicable),  providing  draft  requirements  documents,  submission  of 
SAS  for  draft  review,  all  the  way  through  the  completion  of  the  ASSP,  resulting  in  the 
signed  SAS. 

Essentially,  we  have  five  separate  objectives  that  are  addressed,  which  resulted  in  a  series 
of  questions.  Although  these  specific  questions  will  not  be  included  in  the  survey,  we  will 
provide  for  insight. 

Objective  1:  Are  the  Army  service  acquisition  requirements/reviews  leading  to  desired 
outcomes? 

Objective  2:  Does  the  value  added  through  completion  of  the  requirements/reviews 
exceed  the  resource  outputs  required  to  complete  the  process? 

Objective  3:  Is  proper  training/mentorship  available  prior  to  the  Army  service  acquisition 
requirements/reviews  to  maximize  the  value  of  the  ASSP? 

Objective  4:  Are  commands  capturing  metrics  to  document  the  requirements/reviews 
impacts? 

Objective  5:  Are  commands  taking  proper  measures  to  capture  lessons  learned? 

We  will  not  reveal  names  or  programs  of  the  participants.  This  is  solely  for  research  in 
developing  the  appropriate  question  and  we  are  seeking  feedback  on  the  question  and  the 
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design  of  the  questions.  Your  comments  on  individual  questions  will  provide  valuable 
feedback  and  will  be  incorporated  into  the  final  survey  instrument. 
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APPENDIX  4.  INITIAL  EMAIL  CORRESPONDENCE 


January  27,  2016 

Good  Afternoon, 

My  name  is  Megan  Weidner,  I  am  a  contract  specialist  at  Army  Contracting  Command 
Orlando.  Your  solicitation  (FBO  announcement)  was  provided  to  me  by  Mr.  Bill  Mercer 
from  ASA(ALT).  My  intention  was  to  get  in  touch  with  individuals  working  acquisitions 
that  recently  went  through  the  Army  Services  Strategy  Panel  (ASSP)  process  for  approval 
of  your  Acquisition  Strategy. 

BLUF:  I  am  working  on  a  Graduate  Thesis  at  the  Naval  Postgraduate  School  entitled:  An 
Analysis  of  the  Army  Service  Acquisition  Review  Requirements  and  the  Perceived 
Effectiveness  of  Intended  Improvements.  As  part  of  the  thesis,  my  team  and  I  will  create 
a  survey,  and  although  we  do  not  intend  to  issue  the  survey,  we  will  field  test  to  ensure 
our  questions  are  clear,  as  intended,  effective,  etc.  To  do  so,  we  must  gather  individuals 
who  would  be  able  to  assist  in  the  ‘development’  of  the  survey  by  reviewing  our 
questions. 

We  hope  to  take  no  more  than  10  minutes  or  so  of  your  time  when  our  survey  is  issued  in 
the  March  timeframe. 

If  you  could,  please  let  me  know  if  you  are  willing  to  help  us  out  so  we  can  reach 
graduation.  Your  response  is  greatly  appreciated! 


March  7,  2016 

Good  Afternoon, 

As  we  had  previously  discussed,  you  have  been  identified  by  Mr.  Bill  Mercer  from  the 
office  of  the  DASA(P)  Senior  Service  Manager,  as  a  recent  participant  in  the  Army 
Services  Strategy  Panel  (ASSP)  Process. 

The  reason  you  have  been  sought  you  out  is  that  I  am  nearing  completion  of  my  Masters 
in  Contract  Management  at  the  Naval  Postgraduate  School.  My  team  and  I  (Mike, 
Roxanne  and  T  CC’d  in  email)  are  working  our  Joint  Applied  Project  (Thesis)  on  Army 
service  acquisitions.  The  title  of  our  project  is:  An  Analysis  of  the  Army  Service 
Acquisition  Review  Requirements  and  the  Perceived  Effectiveness  of  Intended 
Improvements. 

Our  thesis  examines  the  existing  policies/guidance  for  service  acquisitions  (OSD,  Army, 
GAO  cases  and  BBP),  analyzes  the  policies/guidance  for  service  acquisitions,  provides 
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the  research  methodology  to  help  determine  the  survey  questions  and  will  conclude  with 
the  findings  and  results.  This  survey  “field  test”  we  have  provided  fits  into  the  research 
methodology  where  we  describe  the  survey  development,  field  testing  procedure  and  will 
analyze  the  provided  results. 

Please  first  review  the  attached  Survey  Instructions,  following  with  the  Survey  Field 
Test.  If  you  happen  to  have  any  questions  during  the  completion  of  the  survey,  please  do 
not  hesitate  to  call  or  email  me.  We  kindly  request  response  by  COB  18  March,  so  that 
we  can  consolidate  the  data  and  incorporate  into  our  thesis. 

We  GREATLY  appreciate  your  time  and  interest  in  supporting  this  matter. 


March  14,  2016 

Happy  Monday! 

In  order  for  our  team  to  adequately  plan  for  receipt  of  survey  responses  and  ensure  we 
receive  a  sufficient  number  of  responses,  I  wanted  to  follow-up  to  the  email  I  sent  last 
Monday  requesting  your  response  and  comments  of  our  ASSP  Process  Survey. 

If  you  intend  to  complete  the  survey,  but  have  been  unable  to  yet,  could  you  please 
provide  a  response  of  your  intentions?  I  have  attached  the  survey  instructions  and 
questions  for  your  reference. 

Again,  we  greatly  appreciate  your  feedback  to  ensure  this  Survey  is  a  success ! 
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